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Aviation  safety  is  most  commonly  measured  by 
accident  rate  vs.  100,00  departures.  Trends, 
depicted  in  Figure  1.1,  show  that  aviation  safety 
benefits  from  continuous  improvement,  meaning  that 
this  earth’s  safest  transportation  is  becoming  even 
safer.  Hardware  is  the  primary  reason  that  aviation 
safety  is  improving.  Modern  power  plants  and 
aircraft  systems  have  increasing  reliability.  Aircraft, 
air  traffic  control,  and  airport  navigation,  landing. 


and  communications  digital  systems  have  also 
contributed  to  the  safety  factor.  Some  suggest  that  the 
extent  to  which  hardware  can  increase  safety  has 
reached  an  assemtote;  it  is  not  likely  to  make  much 
more  improvement.  However,  attention  to  the  human 
as  operator  and  maintainer  of  the  aviation  safety 
system,  has  the  highest  potential  for  additional  safety 
enhancement.  In  fact,  human  error  is  the  #1  cause  of 
aviation  incidents  and  accidents  (NTSB). 


Phase  V  Overview 


Chapter  1 


Since  1989  the  FAA  Office  of  Aviation  Medicine  has 
conducted  research  related  to  human  factors  in 
aviation  maintenance.  The  research  program  is  the 
world’s  largest  such  study  of  human  performance  in 
maintenance.  Involving  universities,  government 
laboratories  and  private  industry,  the  research 
addresses  many  aspects  of  human  performance  in 
maintenance.  The  research  ranges  from  basic 
scientific  experimentation  to  applied  studies  in  airline 
work  environments.  The  applied  studies  represent  the 
largest  part  of  the  program. 

The  human  factors  in  aviation  maintenance  research 
program  uses  airline  and  industry  maintenance 
facilities  as  the  primary  laboratories.  FAA  inspectors 
working  on  airline  air  worthiness  have  also  helped  to 
define,  develop,  and  evaluate  products  of  the  human 
factors  research. 

In  the  six  years  of  the  research,  the  Office  of 
Aviation  Medicine  has  conducted  and  published 
proceedings  of  nine  workshops  on  Human  Factors  in 
Maintenance  and  Inspection.  The  research  team  has 
published  over  200  technical  papers.  Three  CD- 
ROMs  have  been  published  and  distributed  to  over 
3,000  recipients. 

This  report  documents  the  primary  research  and 
development  efforts  conducted  in  the  fifth  year  of  the 
research  program.  As  in  previous  years,  the  report 
represents  a  broad  spectrum  of  human  performance 
research  and  development,  each  shall  be  described 
briefly  in  the  remainder  of  this  introductory  chapter, 

1.1  Job  Aiding  for  Aviation  Safety  Inspectors 
(Chapter  2) 

The  Performance  Enhancement  System  (PENS)  is  an 
ongoing  research  and  development  effort  to  empower 
FAA  Aviation  Safety  Inspectors  (ASIs)  with  mobile 
computing  software  and  hardware.  The  chapter 
describes  two  mobile  computing  applications,  one  for 
government  (PENS)  and  the  other  for  industry 
(CASE). 

PENS  provides  ASIs  with  a  mobile  computer  to 
collect  and  analyze  data  in  the  field.  The  system, 
described  in  the  chapter,  also  permits  ASIs  electronic 
access  to  critical  data  like  the  Federal  Aviation 
Regulations  and  the  FAA  Inspectors  Handbooks.  The 
chapter  also  describes  an  extensive  field  test  of  PENS 
and  ongoing  evaluations  of  emerging  mobile 
computing  hardware  and  software  technology. 


The  airlines  share  a  system  to  audit  providers  of 
goods  and  services.  The  system  is  named 
Coordinating  Agency  for  Supplier  Evaluation 
(CASE).  The  CASE  system  is  comprised  of  paper 
forms  and  a  hard  copy  instruction  guide  book  to 
complete  the  forms.  The  CASE  mobile  computing 
software  has  integrated  all  information  into  a 
complete  digital  system.  The  chapter  2  appendix 
describes  the  CASE  software. 

1.2  Compiiter-based  Trainieg  for  Regulatory 
Documents  (Chapter  3) 

The  System  for  Training  Aviation  Regulations 
(STAR)  combines  multimedia  training  software  and 
the  FAA  Human  Factors  Information  System  (HIS) 
to  provide  a  mix  of  training  and  digital 
documentation.  The  training  system  is  being 
designed  to  present  cases,  or  scenarios,  to  learn  about 
the  Federal  Aviation  Regulations  and  other 
regulatory  documents  for  maintenance.  The  chapter 
describes  how  STAR  instructional  design  and 
training  system  analysis  were  conducted. 

Descriptions  of  STAR  functionality  are  also 
included. 

1.3  Digital  Documentation  Systems  (Chapter  4) 

The  research  program  has  a  rich  history  applied  to 
digital  documentation  systems.  The  Human  Factors 
Information  System  (HIS)  is  a  hypertext  multimedia 
software  system  that  was  developed  for  FAA  CD- 
ROMs  1-3.  This  special  purpose  system  was 
designed  to  meet  specific  FAA  hypertext 
requirements  and  to  minimize  costs  associated  with 
mass  production  and  distribution  of  certain  FAA 
databases.  This  chapter  describes  the  design  and 
evolution  of  HIS.  It  also  shows  interface  examples  of 
how  HIS  is  applied  to  the  CD-ROMs  and  to  the 
digital  Human  Factors  Guide. 

1.4  On-Ramp  to  Information  Superhighway 
(Chapter  5) 

The  Office  of  Aviation  Medicine  has  distributed 
research  results  via  three  CD-ROMs,  as  previously 
described.  This  media  has  worked  well  as  the  number 
of  installed  CD-ROM  computers  has  increased  in 
government  and  throughout  the  aviation  industry. 

The  research  related  to  the  “FAA  Information 
Skyway”  is  developing  the  hardware/software 
infrastructure  to,  eventually,  distribute  research 
results  via  the  Internet. 
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Chapter  1 


Phase  V  Overview 


The  chapter  describes  a  user  assessment  of  the  on¬ 
line  information  needs  of  the  aviation  maintenance 
community.  The  chapter  describes  the  kinds  of 
services  that  are  needed  and  likely  to  be  provided  by 
an  “Information  Skyway.”  The  initial  World-Wide 
Web  has  been  established  and  is  operational.  The 
chapter  describes  the  services/reports  that  are 
currently  available.  It  also  describes  future  directions. 

1.5  Developsnesit  of  an  Airline  Homan  Factors 
Program  (Chapter  6) 

This  project  was  done  in  cooperation  with  Northwest 
Airlines,  at  the  DC-9  base  in  Atlanta.  The  goal  was  to 
establish  a  human  factors  task  force  to  review  a 
variety  of  human  performance  issues  associated  with 
the  inspection  department. 

The  chapter  describes  how  the  task  force  was  formed 
and  the  composition  of  worker  and  management 
participants.  Also  described  are  a  variety  of 
opportunities  for  improvement  in  decision  making 
and  communication  in  the  maintenance  process. 

1.6  An  Audit  System  for  Maintenance  Human 
Factors  (Chapter  7) 

The  purpose  of  this  task  was  to  provide  a  valid, 
reliable,  and  usable  tool  for  evaluating  human  factors 
in  maintenance  tasks.  A  software  tool  was  designed 
and  developed  as  a  product  of  this  research.  As 
reported  in  the  chapter  the  majority  of  the  work  went 
towards  the  ergonomics  audit  information  with  the 
software  development  task  being  secondary.  The 
chapter  includes  hard  copies  of  most  of  the  forms 
contained  in  the  software  program.  The  final  version 
of  the  ergonomics  software  package  shall  be  included 
with  the  digital  publication  of  the  Human  Factors 
Guide. 

1.7  Checklist  Reliability  (Chapter  8) 

Maintenance  workcards  are  the  technician’s 
equivalent  of  the  pilot’s  checklist.  The  workcard  is 
meant  to  ensure  that  maintenance  is  performed  in  the 
correct  order  and  that  no  step  is  omitted.  The  chapter 
reports  on  a  study  of  how  the  design  of  workcards 
affects  their  use  and  the  subsequent  potential  for 
error. 

The  chapter  describes  a  task  analysis  of  workcard 
usage  conducted  in  an  airline  maintenance 
environment.  The  research  analyzed  maintenance 
data  from  the  Aviation  Safety  Reporting  System  to 
determine  if  workcard  usage  or  non-usage 


contributed  to  safety  infractions.  Also  reviewed  is 
application  literature  on  human  error  with  respect  to 
checklists.  The  chapter  ends  with  a  description  of  the 
creation  and  evaluation  of  a  workcard  for  shift 
turnover. 

1.8  Cooperative  Work  with  Aging  Aircraft 
Inspection  Validation  Center  (Chapter  9) 

The  Office  of  Aviation  Medicine  has  engaged  in 
cooperative  research  with  the  FAA  Technical  Center 
via  the  Aging  Aircraft  Inspection  Validation  Center 
(AANC).  The  research  supports  the  Visual  Inspection 
Research  Program  at  Sandia  National  Laboratories  in 
Albuquerque,  NM.  The  chapter  describes  the  process 
of  visual  inspection  and  describes  an  evaluation 
measuring  visual  inspection  performance. 

1.9  Individual  Differences  in  Inspection 
Performance  (Chapter  10) 

Numerous  research  studies  have  shown  a  wide  range 
of  individual  performance  differences  among 
inspection  personnel.  This  basic  scientific  study 
measures  relationships  between  NDI  task 
performance  and  psychometric  measures  of 
mechanical  ability  and  attention-concentration.  The 
chapter  describes  a  battery  of  mechanical  aptitude 
tests,  a  simulated  NDI  task,  and  the  ability  of  the  tests 
to  predict  performance.  The  exciting  answer  to  these 
predictive  questions  can  be  found  in  the  chapter! 

1.10  Study  of  Teamwork  in  Maintenance 
(Chapter  11) 

Most  maintenance  activities  are  conducted  by  teams 
of  aviation  maintenance  technicians  (AMTs). 
Therefore,  team  planning,  coordination,  and 
communication  are  critical  to  safe  and  efficient 
completion  of  all  maintenance  tasks.  This  chapter 
reports  on  a  study  of  teamwork  in  maintenance  and 
outlines  a  training  program  focusing  on  teamwork. 
The  chapter  reports  the  results  of  an  evaluation  of  a 
teamwork  training  program  conducted  in  a  FAR  147 
school.  The  chapter  ends  with  a  technical 
specification  for  a  computer-based  training  system 
for  team  training. 

1.11  Advanced  Certification  Initiatives 
(Chapter  12) 

FAR  65  addresses  the  certification  of  aviation 
personnel  other  than  Right  crew  members.  Over  the 
past  few  years  the  FAA,  in  cooperation  with  an 
Aviation  Rulemaking  Advisory  Committee  (ARAC), 
has  been  revising  Part  65  to  address  competencies 
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Chapter  1 


and  requirements  for  Aviation  Maintenance 
Technicians.  This  chapter  reports  on  the  ARAC 
activities  and  impending  rule  changes.  This  chapter 
also  considers  methods  to  create  an  “advanced 
certification”  system  that  could  be  administered  by 
private  industry  instead  of  FAA. 

lol2  HuBmaiini  Factors  Workshop- Appendices 

The  Office  of  Aviation  Medicine  has  conducted  nine 
workshops  on  Human  Factors  in  Maintenance  and 
Inspection.  The  proceedings  from  eight  of  these 
workshops  are  published  in  hard  copy  and  on  the 
FAA  CD-ROMs.  The  ninth  conference  was  held  in 
November,  1994,  and  focused  on  review  of  the 
Human  Factors  Guide  for  Aviation  Maintenance. 

Few  speakers  at  the  ninth  meeting  spoke  on  topics 
other  than  specific  chapters  of  the  Guide.  Therefore, 
a  dedicated  9th  Meeting  Proceedings  shall  not  be 
published. 

The  appendices  of  this  report  contain  papers  from  the 
9th  meeting  that  are  not  directly  related  to  the  Human 
Factors  Guide.  The  first  speaker  was  Dr.  Jon  L. 
Jordan,  Federal  Air  Surgeon.  Dr.  Jordan’s  paper 
reviewed  the  five  year  progress  of  the  research 


program.  He  highlights  major  program  products  and 
looks  to  the  future  of  the  research  program. 

Dr.  Patrick  Walter  is  the  Director  of  the  Aging 
Aircraft  Inspection  Validation  Center  at  Sandia 
National  Laboratory.  His  paper  describes  the  research 
program  at  Sandia.  The  appendix  also  contains  a 
paper  from  Mr.  Eddie  Rogan,  Human  Factors 
Engineer  -  British  Airways.  Mr.  Rogan  describes  the 
human  factors  research  at  British  Airways  with 
specific  reference  to  the  Managing  Engineering 
Safety  Health  (MESH)  system.  MESH  is  a  method 
for  reporting,  analyzing,  and  mitigating  human  error 
in  maintenance. 

Also  included  in  the  appendices  is  a  list  of  attendees 
who  participated  in  the  Agenda  9th  Workshop. 


NTSB,  Broad  Cause/Factor  Assignments,  14  CFR 
121  Operations,  1992 
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Job  Aiding: 

Performance  Enhancement  System 


Charles  Layton,  Ph.D. 
Galaxy  Scientific  Corporation 


One  of  the  tasks  in  the  Human  Factors  in  Aviation 
Maintenance  and  Inspection  Research  Program 
involves  investigating  advanced  technologies  and 
how  these  technologies  might  be  applied  to  aviation 
maintenance  tasks.  We  have  been  investigating  pen 
computing  technology  and  have  developed  a 
prototype  application,  called  the  Performance 
Enhancement  System  (PENS),  for  the  FAA  Flight 
Standards  Service.  We  have  also  been  working  on  a 
transition  of  our  experiences  from  this  project  to 
industry.  The  bulk  of  this  chapter  describes  the  Flight 
Standards  work,  while  Qiapter  2  -  Appendix 
addresses  the  work  we  have  done  with  an  industry 
partner. 

We  had  several  milestones  with  PENS  in  the  last 
year.  The  first  field  study  was  completed  in  April 
1994,  and  the  results  of  that  study  were  published  last 
fall.  Fall  1994  also  saw  the  initiation  of  FAA  training 
of  Aviation  Safety  Inspectors  on  PENS  concepts. 
Version  2  of  the  system  software  was  completed  in 
preparation  for  a  second  field  study  in  Winter 
1994/1995.  Finally,  a  number  of  computers  have 
been  evaluated  in-house,  and  several  units  have  been 
selected  for  in  the  study  to  evaluate. 


2.1  BAQKGROUNa. 


The  Performance  Enhancement  System  represents 
a  series  of  investigation  and  implementation 
phases  supporting  the  goal  of  matching  the  needs  and 
responsibilities  of  Right  Standards  Service  (AFS) 
Aviation  Safety  Inspectors  (ASIs)  with  automation 
capabilities.  This  project  is  a  direct  result  of  the  AFS 
Training  and  Automation  Committee’s  Information 
Systems  Strategy,  which  recommended  that  all  future 
automation  systems  be  developed  in  conjunction  with 
the  work  force  so  that  systems  are  designed  to  meet 
workers’  needs  and  desires.  The  Training  and 
Automation  Committee  has  been  instrumental  in 
supporting  PENS  and  in  providing  project  oversight. 


Field  data  collection  is  one  characteristic  of  ASI 
activities.  The  data  are  collected  on  paper  forms,  and 
data  entry  clerks  transcribe  these  forms  into  computer 
databases.  These  data  are  then  recorded  in  a  national 
database  and  are  used  to  monitor  the  aviation 
industry’s  safety.  Another  characteristic  of  field 
inspectors’  activities  is  that  they  must  authoritatively 
answer  questions  as  they  arise.  This  requires  ASIs  to 
carry  voluminous,  cumbersome  field  copies  of 
regulations  and  guidance. 

Four  primary  concerns  provided  the  impetus  for 
development  of  PENS.  First,  data  entry  clerks  are  a 
significant  annual  expense  for  AFS.  If  it  were  easy 
for  inspectors  to  enter  data  into  the  computer 
databases  themselves,  AFS  would  save  the  money  it 
now  spends  on  data  entry.  Second,  there  is  a 
significant  time  delay  of  up  to  two  weeks  in  form 
transcription.  By  decreasing  that  time  delay,  AFS 
could  be  more  effective  at  monitoring  and  ensuring 
compliance  in  the  aviation  industry.  Third,  many  data 
transcription  errors  occur  in  the  current  process,  so 
many  that  the  Government  Accounting  Office  has 
repeatedly  criticized  the  FAA  for  the  poor  quality  of 
its  data.  Fourth,  paper  regulations  and  guidance 
materials  are  not  used  effectively  because  they  are 
bulky  and  difficult  to  maintain.  The  combination  of 
all  these  factors  points  toward  automation  as  a 
potential  solution.  Field  automation,  at  a  minimum, 
would  allow  ASIs:  1)  to  store  data  directly  in  the 
proper  database  format;  2)  to  verify  the  validity  of 
data  at  the  time  of  an  inspection;  3)  to  eliminate  the 
time  delay  associated  with  transcription;  and  4)  to  use 
on-line  guidance  materials  quickly,  easily,  and  with 
minimal  maintenance  of  the  documents.  Other 
benefits  would  accrue  as  more  tools  were  added  to 
field  computers. 

The  project  began  as  an  investigation,  sponsored  by 
the  Office  of  Aviation  Medicine  (AAM),  into  the 
utility  of  pen  computers  for  aviation  industry 
inspectors  and  maintenance  technicians.  This  phase 
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of  the  project  continued  from  approximately  January 
until  August  1992.  During  this  time,  FA  A 
Administrator  Thomas  Richards  learned  about  pen 
computers  and  thought  that  they  might  be  a  good  tool 
for  Aviation  Safety  Inspectors.  To  this  end,  he 
requested  briefings  from  the  Flight  Standards 
Service.  The  Flight  Standards  Service  learned  of  the 
AAM  research  and  requested  information  in  August 

1992.  After  a  series  of  briefings  to  FAA  personnel, 
including  Clyde  Jones,  AFS  Director  Thomas 
Accardi,  and  Associate  Administrator  for  Regulation 
and  Certification  Anthony  Broderick,  we  briefed 
Administrator  Richards  in  November  1992,  and 
Acting  Administrator  Joseph  Del  Balzo  in  January 

1993. 

Between  January  and  August  1993,  PENS  received  a 
lot  of  publicity  within  Flight  Standards  Services,  both 
in  AFS  Headquarters  and  in  the  field.  The  project 
continued  with  a  low  level  of  funding  from  the  Office 
of  Aviation  Medicine.  From  August  1992  through 
August  1993,  a  series  of  task  analyses  and  prototypes 
were  carried  out  to  determine  the  basic  content  of  a 
field  computer  tool.  The  Fort  Lauderdale  Flight 
Standards  District  Office  (FSDO)  was  fundamental  to 
the  success  of  these  initial  analyses  and  prototypes. 

Funding  for  a  national  field  human  factors  study  of 
PENS  concepts  was  provided  in  August  and  October 
of  1993.  Because  of  all  of  the  publicity  the  project 
had  received  over  the  previous  year,  AFS 
Headquarters  felt  considerable  pressure  to  start  the 
field  study  quickly  once  funding  was  available.  After 
some  very  rapid  prototyping  and  testing  with  Atlanta 
FSDO  inspectors,  the  national  field  study  began  on 
November  15,  1993,  continuing  until  March  1,  1994. 


The  following  is  a  summary  of  Performance 
Enhancement  System  concepts  that  were 
evaluated,  the  nature  of  the  field  study,  the  important 
results,  and  considerations  for  full  implementation. 
The  full  results  and  discussion  can  be  found  in  The 
Performance  Enhancement  System  Field  Evaluation 
Report, 

2*2.1  Inspector  Characteristics 

Four  airworthiness  (maintenance)  aviation  safety 
inspectors  at  each  of  nine  sites,  a  total  of  36 
inspectors,  participated  in  the  study.  The  inspectors 
averaged  49  years  in  age,  had  been  inspectors  for  five 


and  a  half  years  (most  airworthiness  inspectors  are 
former  aircraft  mechanics),  and  had  five  and  a  half 
years  of  computer  experience.  Sixty-five  percent  of 
the  inspectors  use  the  current  data  entry  system,  and 
sixty  percent  own  computers. 

Note  that  inspectors’  computer  experience  correlates 
with  their  experience  as  ASIs.  The  current  computer 
systems  installed  at  the  field  evaluation  sites  run  a 
very  limited  set  of  DOS  applications,  not  Microsoft 
Windows  applications.  PENS  runs  in  Microsoft 
Windows  for  Pen  Computing. 

Training  was  given  according  to  time,  rather  than  to 
criterion.  Inspectors  were  trained  for  two  days.  The 
first  day  consisted  of  an  explanation  of  file  storage 
conventions,  DOS,  Windows,  and  handwriting 
recognition,  including  training  the  computer  to 
recognize  the  inspectors'  handwriting.  The  second 
day  consisted  of  training  on  PENS  software. 

We  spent  much  more  time  covering  basics  in 
Windows  than  we  thought  would  be  necessary.  Even 
though  each  office  had  Windows  installed  on  its 
workstations,  inspectors  were  generally 
inexperienced  Windows  users.  The  most  likely 
explanation  for  their  inexperience  was  that  few 
inspectors  had  any  need  to  run  Windows  software. 
The  extra  Windows  training  did  not  significantly 
affect  the  amount  of  training  devoted  to  PENS;  there 
was  time  left  at  the  end  of  the  second  training  day. 

2.2.2  Materials 

Three  different  models  of  pen  computers  and  one 
standard  notebook  computer  were  fielded  at  each 
office.  Thus,  36  computers  were  put  into  the  field. 
Computers  were  selected  based  on  their  particular 
combination  of  features  and  their  differentiating 
characteristics.  That  is,  the  computers  were  selected 
because  they  had  certain  features  in  common,  but 
each  also  had  a  particular  feature  that  made  it  unique. 
These  computers  allowed  inspectors  to  evaluate  the 
tradeoffs  between  weight,  versatility,  and  speed.  The 
computers’  features  are  summarized  in  Table  2.1 
(next  page).  The  features  listed  in  Table  2.2  (next 
page)  are  common  to  all  four  computers. 

2.2.3  Results—Competer  Platforms 

The  inspectors  were  asked  to  rate  a  number  of 
usability  characteristics  of  each  computer.  The 
characteristics  included  weight,  ease  of  use,  screen 
characteristics,  environments  in  which  the  computer 
was  used,  and  the  like.  With  regard  to  particular 


6 


Chapter  2 


Job  Aiding:  PENS 


GRiD  Convertible 
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TelePad  SL 

Toshiba  Satellite  T1900 

486/25  MHz  CPU 

486/25  MHz  CPU 
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486/25  MHz  CPU 

200  Mb  Hard  Drive 

80  Mb  Hard  Drive 

200  Mb  Hard  Drive 

120  Mb  Hard  Drive 

Built-in  Keyboard 

Separate  Keyboard 

Separate  Keyboard 

Built-in  Keyboard 

Pen  Stylus 

Pen  Stylus 

Pen  Stylus 

Trackball 

8  Mb  RAM 

Backlit  LCD  Monochrome  display 
PCMCIA  Data  Storage  Card 
DOS  6.0 
Windows 

Microsoft  Word  2.0  (except  the  NEC  VersaPad) 
PENS  Prototype  Software _ 


characteristics  of  pen  computers,  the  only  significant 
result  was  that  the  GRiD  Convertible  was  judged 
more  comfortable  than  the  NEC  VersaPad.  This 
result  is  consistent  with  inspectors’  comments  that  its 
case  made  the  VersaPad  difficult  and  cumbersome; 
the  Convertible  was  much  more  compact  and  easy  to 
use. 

When  ratings  for  pen  computers  are  compared  with 
the  notebook  computer  (Toshiba  Satellite  T1900), 
both  the  GRiD  Convertible  and  the  TelePad  SL  were 
judged  to  be  faster.  Inspectors  generally  disliked  the 
VersaPad,  and  that  may  have  biased  the  inspectors’ 
evaluations.  We  originally  thought  that  the  VersaPad 
was  a  good  computer  to  use  to  examine  tradeoffs 
between  computer  characteristics  because  it  had  a 
smaller  hard  disk  and  was  also  much  lighter. 

Finally,  inspectors  addressed  the  tradeoff  between 
weight  and  capability.  Many  inspectors  complained 
that  the  VersaPad  did  not  have  enough  hard  disk 
capacity  because  it  was  too  small  to  contain  on-line 
versions  of  both  the  FARs  and  the  Airworthiness 
Inspectors’  Handbook. 

Perhaps  the  most  telling  data  on  the  computers  were 
collected  in  response  to  the  question,  “Would  you  use 
this  computer  in  the  field  as  part  of  your  job?” 
Inspectors  generally  preferred  the  GRiD  Convertible 
and  the  TelePad  SL  over  the  NEC  VersaPad  and  the 
Toshiba  Satellite.  However,  none  of  these  computers 
are  currently  in  production:  the  GRiD  Convertible 
and  the  NEC  VersaPad  have  been  removed  from  the 
market;  the  TelePad  SL  is  due  to  be  replaced  this  Fall 
with  the  TelePad  3;  and  the  Toshiba  Satellite  T1900 
has  been  replaced  with  another  model. 


Because  the  notebook  computer  was  comparatively 
heavy  and  cumbersome,  it  was  extremely  difficult  for 
inspectors  to  use  it  while  they  performed  an 
inspection.  While  they  could  easily  operate  a  pen 
computer  with  two  hands,  the  notebook  computer 
really  needed  to  lie  on  a  flat  surface.  Inspectors 
indicated  that  they  definitely  would  not  be  able  to  use 
a  standard  notebook  computer  as  part  of  their  daily 
routine,  although  a  pen  computer  was  feasible. 

Inspectors  were  unanimous  in  requesting  smaller, 
lighter  computers.  They  were  particularly  interested 
in  devices  that  would  fit  in  their  coat  pockets  such  as 
personal  digital  assistants,  e.g.,  Apple  Newton, 
Tandy/Casio  Zoomer,  etc.  However,  such  devices 
currently  do  not  have  either  the  storage  or  the 
processing  resources  to  run  applications  necessary  for 
ASIs.  Inspectors  were  also  intrigued  by  the 
possibility  of  using  speech  recognition  for  data 
collection,  as  this  would  keep  their  hands  free. 

2.2.4  Additional  Issues 

Interviews  with  inspectors  revealed  that,  although 
immediate  recording  of  field  data  may  not  always  be 
required,  immediate  access  to  previous  data  or 
regulatory  materials  is  required.  For  inspectors,  a 
computer  is  more  useful  as  an  information 
management  and  retrieval  tool  than  as  a  data 
collection  vehicle  for  inspection  activities. 

Inspectors  raised  a  number  of  additional  concerns 
during  the  study.  Many  inspectors  were  concerned 
about  liability  for  the  equipment  should  it  be  stolen, 
dropped,  or  left  on  an  airplane.  Some  inspectors  were 
concerned  with  perceptions  of  people  they  were 
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inspecting,  i.e.,  they  were  worried  that  they  appeared 
inept  or  incompetent  when  using  a  computer.  Other 
inspectors  were  concerned  that  a  computer  lent  an  air 
of  permanence  to  notes  they  made,  and,  as  a  result, 
operators  would  be  less  cooperative,  even  though 
notes  on  paper  have  the  same  degree  of  permanence. 
While  there  are  practical  solutions  to  all  these  issues, 
the  issues  themselves  go  well  beyond  the  questions  of 
which  computer  is  better  or  if  a  field  computer  can  be 
used  for  one-time  data  capture. 

With  regard  to  environmental  considerations, 
inspectors  noted  that  the  computers  stopped  working 
when  the  temperature  approached  freezing.  Cold 
temperatures  also  make  it  more  difficult  to  use  a 
computer  because  of  the  inspector’s  need  to  wear 
gloves,  bulky  coats,  etc.  Finally,  as  one  might  expect, 
inspectors  were  reluctant  to  use  computers  in  snow  or 
rain  for  fear  of  damaging  the  machines. 


The  Regulatory  Standards  and  Compliance 
Division,  AMA-200,  has  begun  training  new 
ASIs  on  the  concepts  embodied  in  the  Performance 
Enhancement  System.  Although  the  system  is  not 
ready  for  full  implementation,  inspectors  should  be 
initiated  into  future  system  capabilities  as  they 
receive  their  first  training.  In  this  way,  inspectors  will 
see  the  system  as  a  tool  in  their  compliance  arsenal 
and  as  an  integral  part  of  their  jobs. 

Version  2  of  the  software  was  only  recently 
completed,  so  the  training  group  has  provided  only  a 
brief  system  introduction  during  the  training  courses. 
However,  the  training  group  has  indicated  that  they 
will  gladly  incorporate  more  training  as  soon  as  the 
system  is  ready  for  full  implementation. 


Version  2  of  the  Performance  Enhancement 
System  software  has  been  completed  and  is 
ready  for  the  next  field  study.  This  software 


incorporates  changes  and  improvements  over  the  last 
version  in  four  major  areas: 

1 .  the  code  was  converted  from  C/C+-I-  to 
Microsoft  Visual  Basic  to  allow  significant 
improvements  in  the  software’s  design  and 
maintainability 

2.  the  software  has  greatly  expanded  its 
functionality  to  address  all  three  ASI 
specialties:  Operations,  Airworthiness,  and 
Avionics 

3.  the  Program  Tracking  and  Reporting 
Subsystem  (PTRS)  data  collected  have  been 
subjected  to  the  same  validation  procedures 
used  on  data  entered  through  the  Flight 
Standards  Automation  System  (FSAS) 

4.  the  three  leading  FAA  digital  regulatory 
guidance  document  systems  will  be 
compared  in  the  field  study. 

The  following  sections  address  each  of  these  areas. 

2AA  Softvrare  Conversion  to  Viseal  Basic 

One  of  the  biggest  changes  in  Version  2  is  that  it  has 
been  converted  from  C/C++  to  Visual  Basic,  which  is 
rapidly  becoming  the  standard  development 
environment  for  Microsoft  Windows  software.  This 
switch  has  improved  the  “look  and  feel”  of  the 
software,  has  made  development  easier,  has  increased 
maintainability,  has  improved  our  ability  to  add 
functionality,  and  has  improved  database  capabilities. 

The  enhancements  in  Version  2  improve  usability 
and  user  acceptance.  As  shown  in  Figures  2.1  and  2.2 
(next  page),  the  scroll  bar  has  been  removed  from  the 
PTRS  form  and  has  been  replaced  with  tabs.  This 
change  makes  navigation  between  sections  of  the 
form  easier  and  more  direct.  Forms  generally  have 
more  visual  depth,  appearing  three  dimensional.  This 
new  appearance  facilitates  functional  grouping  and 
makes  buttons  distinct  from  fields.  Version  2  gives 
users  the  impression  that  it  is  a  professional  product, 
rather  than  a  research  and  development  tool. 
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Because  many  development  tasks  are  handled  by 
Visual  Basic,  rather  than  by  a  programmer,  software 
development  has  become  much  easier.  Since  the 
programmer  does  not  have  to  worry  about  low  level 
Windows  routines  necessary  to  make  buttons  work, 
he  or  she  can  focus  on  greater  design  issues  of  layout, 
error  prevention,  database  support,  and  the  like. 
Furthermore,  Visual  Basic  improves  Version  2’s 
maintainability  because  it  is  now  much  easier  to 
follow  the  software’s  flow  of  control  and  structure. 
Since  Visual  Basic  uses  the  Basic  programming 
language  (which  is  frequently  the  first  computer 
language  one  learns)  the  odds  that  the  FAA  will  be 
able  to  maintain  the  software  are  greatly  improved — 
especially  when  Visual  Basic  is  compared  with  an 
esoteric  language  like  C  or  C++. 

Visual  Basic  supports  myriad  control  features 
allowing  one  to  add  features  supporting  specific 
requirements  of  an  application.  These  controls  are 
called  VBXs,  and  many  are  supplied  by  Microsoft 
with  Visual  Basic.  Thousands  more  are  available 
from  third  parties.  Had  the  project  been  continued  in 
C/C++,  these  types  of  controls  would  have  been 
developed  in-house,  requiring  significant  time  and 
effort.  In  Version  1  of  PENS,  virtually  any  desired 
control  outside  the  very  limited  set  supported  by  the 
C/C++  compiler  would  have  to  have  been  developed 
from  scratch. 

Finally,  Visual  Basic  includes  database  support  for  a 
variety  of  databases,  including  Microsoft  Access  and 
Paradox  3.5.  This  support  allows  us  easily  to  migrate 
the  software  to  support  future  databases  as  AFS 
systems  evolve.  The  current  AFS  standard  database 
format  is  Paradox  3.5,  but  it  appears  that  in  the  near 
future  Microsoft  Access  and  SQL  formats  will  be 


used.  Visual  Basic  has  built-in  support  for  each  of 
these  formats. 

2.4.2  Expaiiraded  Software  Capabilities 

Version  1  of  PENS  consisted  of  three  primary 
modules:  the  data  collection  and  on-line  policy 
module;  the  data  transfer  module;  and  the 
supervisory  review  module.  Each  module  and  its 
improved  version  is  discussed  in  turn. 

The  data  collection  and  on-line  policy  module 
consisted  of  the  PTRS  form  for  data  collection,  the 
Federal  Aviation  Regulations  (FARs),  and  the 
Airworthiness  Inspector’s  Handbook  (FAA  Order 
8300.10).  Version  2  of  this  module  has  been  split  into 
its  constituent  parts.  The  data  collection  portion  has 
been  expanded  to  include  the  ten  forms  most 
commonly  used  in  the  field  (not  in  the  office), 
including  the  PTRS  form.  These  ten  forms  address 
the  operations  and  avionics  specialties,  in  addition  to 
airworthiness. 

New  data  management  capabilities  have  been 
designed  into  Version  2.  Work  has  been  divided  into 
three  general  categories:  work  yet  to  be  begun  resides 
in  the  “In  Box”;  work  started,  but  incomplete,  resides 
in  ‘Work  in  Progress”;  the  “Out  Box”  contains 
completed  activities  before  they  are  transferred  to  the 
office  databases.  A  fourth  data  repository,  the 
“Archive,”  maintains  a  backup  set  of  all  data  that 
have  ever  resided  on  the  portable  computer.  With  this 
structure,  inspectors  quickly  determine  what 
activities  are  currently  open,  what  activities  are 
completed,  and  what  activities  remain  to  be 
accomplished.  This  capability  is  illustrated  in  Figure 
2.3  (next  page). 
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Extensive  error  prevention  mechanisms  have  been 
built  into  these  forms.  The  philosophy  of  the  PENS 
design  process  is  to  guide  users  so  that  they  enter 
correct  data,  not  to  correct  errors  after-the-fact. 
Wherever  possible,  databases  have  been  incorporated 
to  allow  the  user  to  select  from  a  set  of  possible 
entries,  rather  than  to  generate  his  or  her  own  entries. 
Data  that  can  be  infen'ed  from  previous  entries  are 
automatically  entered  into  the  forms.  For  example, 
values  for  the  Callup,  Start,  and  Completion  Dates 
are  constrained  by  the  inspection’s  status.  As  shown 
in  Figure  2.3,  the  “Start  Date”  field  is  grayed  because 
the  Status  is  “P”  for  planned.  Once  the  Status  is  “O” 
for  open,  the  “Start  Date”  field  is  immediately 
available.  Finally,  data  that  are  redundant  across 
forms  are  automatically  shared  so  that  an  inspector 
need  record  those  data  only  once. 

The  on-line  help  system  has  been  expanded  to 
include  Version  2’s  new  functional  capabilities.  Help 
nov/  addresses  how  to  use  the  software,  rather  than 
how  to  complete  a  given  activity.  However,  steps  to 
complete  an  activity  will  be  included  in  Version  3  of 
the  software  because  Job  Task  Analyses  are  to  be 
incorporated.  Two  additional  help  i'eaturcs  have  also 
been  incorporated  in  Version  2:  Bubble  Help  and 
Micro  Help.  Bubble  Help  is  familiar  to  most 
Microsoft  sol’twaire  product  users;  it  is  the  text 


description  appearing  when  the  pointer  rests  on  an 
icon.  Bubble  Help  ensures  that  toolbar  functionality 
is  clear.  Bubble  Help  is  illustrated  in  Figure  2.4  (next 
page).  Micro  Help  is  a  text  description  of  the 
function  currently  in  use  appearing  at  the  bottom  of 
the  screen.  For  example,  when  a  user  clicks  on  the 
“Make-Model-Series”  field  in  the  PTRS  form.  Micro 
Help  indicates  that  the  code  may  be  selected  from  a 
list.  Micro  Help  is  shown  in  Figure  2.5  (next  page). 

The  on-line  FARs  and  Handbooks  in  Version  1  were 
very  difficult  to  maintain  and  keep  current.  Because 
some  commercial  vendors  specialize  in  such 
documents,  it  was  deemed  appropriate  that  inspectors 
compare  the  most  promising  of  commercial 
alternatives.  The  in-housc  versions  of  these 
documents  arc  not  incorporated  in  Version  2.  This 
topic  is  discussed  in  more  detail  below. 

The  data  transfer  module  has  been  divided  into  two 
separate  utilities  in  Version  2.  One  of  these  utilities 
transfers  FSAS  data  to  the  field  computer;  the  other 
transfers  data  from  the  field  computer  to  FSAS.  The 
Ibrmer  utility  will  be  used  rarely,  for  example  when  a 
Held  computer  is  initially  loaded  with  the  inspector's 
work  program.  The  inspector  will  use  the  latter  utility 
whenever  he  or  she  returns  from  the  Held  and  is 
ready  to  transfer  Held  data  tt^  the  office  file  server. 
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The  supervisory  review  module  has  been  dropped 
from  Version  2  because  inspectors  rarely  used  it  in 
the  first  field  evaluation. 

2.4.3  PTRS 

Data  Validation,  the  Regulatory  Support  Division, 
AFS-600,  and  the  Operational  Systems  Branch,  AFS- 
620,  in  particular,  have  been  instrumental  in  allowing 
us  to  test  the  PTRS  data  collection  software.  The 
Operational  Systems  Branch  initiated  a  procedure 
that  allows  us  to  send  PTRS  data  collected  with  our 
software  through  the  same  upload  procedure  utilized 
in  FSDOs,  including  data  validation.  This  allows  us 
to  ensure  that  all  data  are  consistent  with  the  current 
FSAS  data  entry  system.  With  Version  1,  we  had 
difficulties  with  some  hidden  database  fields  our 
software  did  not  fill  and  we  were  unaware  of  these 
difficulties  until  we  started  field-testing  the  software. 
Version  2’s  data  validation  capability  allows  us  to 
work  out  such  kinks  before  we  get  the  software  into 
the  field. 

2.4.4  Digital  Regulatory  Guidance  Documents 

As  noted  above,  one  of  the  critical  needs  inspectors 
cited  in  the  first  field  study  is  an  ability  to  research 
policy  and  regulatory  guidance  while  they  are  in  the 
field.  Version  1  of  the  software  supported  a  prototype 
of  this  capability.  At  the  time,  it  was  necessary  for  us 
to  develop  this  prototype  in-house  because  the 
products  were  not  available  commercially.  However, 
three  commercial  providers  now  have  released 
extensive  Windows-based  systems:  Aviation 
Compliance  Services  (ACS)  released  the  FAR 
Library;  Aircraft  Technical  Publishers  (ATP) 
released  the  United  States  National  Aviation 
Regulatory  Library;  and  Summit  Aviation  released 
the  Computerized  Aviation  Publications  Library. 

Each  system  contains  the  Federal  Aviation 
Regulations,  some  Advisory  Circulars,  some  FAA 
Orders,  and  additional  publications.  Each  package  is 
unique,  and  each  publisher  releases  updates  on  its 
own  schedule. 

The  ACS  and  Summit  systems  have  a  simple 
document  viewer  with  simple  searching  techniques. 
The  ATP  system  is  a  powerful  research  tool, 
containing  significant  cross  referencing  of  documents 
and  aircraft  information.  There  are  significant  cost 
differences  among  the  products.  Our  current  plan  is 
to  compare  all  three  products  in  a  small  field  study 
and  then  to  let  inspectors  determine  which  product 
best  meets  their  needs.  ACS  and  ATP  have  agreed  to 


supply  their  product  at  cost;  negotiations  with 
Summit  are  underway. 


We  are  continuing  to  evaluate  portable  computers 
to  stay  abreast  of  the  latest  developments  in 
portable  computing  technology.  Portable  computers 
are  becoming  smaller  and  lighter,  with  more 
processing  power,  and  a  longer  battery  life.  New 
developments  in  pen  computer  technology  have 
allowed  manufacturers  to  reduce  their  size  and 
weight  while  simultaneously  increasing  their 
capabilities  and  battery  life.  These  units  have 
improved  so  much  recently  that  they  deserve  a  fresh 
look  from  inspectors,  particularly  from  airworthiness 
inspectors. 

Subnotebook  computers  offer  a  compromise  between 
the  capabilities  of  full  notebook  computers  and  their 
weight.  Subnotebooks  typically  have  somewhat 
smaller  hard  disk  drives  of  around  1 20  MB  (although 
this  is  increasing)  and  use  external  floppy  drives; 
they  are  much  smaller  than  notebook  computers  and 
weigh  approximately  half  as  much.  A  subnotebook 
computer  will  fit  in  a  large  overcoat  pocket,  which 
approaches  inspectors’  requests  for  a  unit  that  would 
fit  in  a  pocket. 

While  subnotebook  computers  may  fit  a  majority  of 
inspectors’  needs,  inspectors  may  also  wish  to  do 
research  on  policy  guidance  in  the  field.  In  the  last 
year  several  notebook  computers  with  internal  CD 
ROM  drives  have  been  introduced.  These  CD  ROM 
notebooks  have  full  multimedia  capabilities,  as  well. 
These  machines  come  in  two  configurations.  One 
design  has  a  CD  ROM  drive  underneath  its  keyboard; 
the  other  uses  a  separate  CD  ROM  docking  station 
attached  beneath  a  standard  notebook  computer.  The 
first  design  has  CD  ROM  available  always;  its 
drawback  is  that  the  user  must  always  carry 
additional  weight.  The  second  design  has  the  merit  of 
allowing  an  inspector  to  leave  the  CD  ROM  drive 
(and  its  weight)  behind  when  it  is  not  needed;  its 
drawback  is  that  an  inspector  has  to  keep  track  of  a 
second  piece  of  equipment. 

We  envision  providing  samples  of  these  computers  to 
inspectors  at  the  Atlanta  FSDO  prior  to  the  actual 
field  study.  These  inspectors  will  give  us  a  first  pass 
evaluation  of  the  options;  in  turn,  we  can  determine 
which  computers  offer  the  most  promise  for  the  field 
study. 
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Chapter  2  -  Appendix 

Job  Aiding:  Transition  of  Performance  Enhancement  System 
Concepts  to  Industry 


The  Performance  Enhancement  System’s  success 
has  brought  the  aviation  industry’s  attention  to 
the  possibilities  of  supporting  mobile  maintenance 
technicians  and  auditors  with  portable  computing 
technology.  This  is  somewhat  ironic,  given  that  we 
started  the  research  with  these  applications  in  mind 
but  were  unable  to  interest  industry.  During  the  last 
year,  we  have  been  working  with  a  partner  airline  to 
transition  PENS  job  aiding  concepts  to  industry 
personnel.  The  following  is  a  brief  description  of  that 
work. 

Airline  Partner’s  Needs 

Our  partner  airline  has  two  groups  of  maintenance 
auditors  within  the  Technical  Standards  office: 
Compliance  Auditors  and  Vendor  Surveillance 
Analysts.  Both  groups  use  a  variety  of  forms  to 
document  the  results  of  their  audits.  Both  groups  also 
have  standards  which  they  apply  to  the  organizations 
that  they  audit,  including  Federal  regulations  (Federal 
Aviation  Regulations,  Airworthiness  Directives,  etc.) 
and  internal  standards.  Our  partner  airline  wanted  to 
support  both  groups  of  auditors. 

The  Vendor  Surveillance  group  is  responsible  for 
auditing  companies  supplying  materials  and  services 
to  the  airline  to  ensure  that  those  companies  are  in 
compliance  with  Federal  guidelines  and  with  industry 
standards.  Our  partner  airline  is  a  member  of  the 
Coordinating  Agency  for  Supplier  Evaluations 
(CASE).  The  CASE  organization  is  a  consortium  of 
airlines  that  pool  their  resources  and  auditing  results. 
If  a  CASE  member,  e.g.,  our  partner  airline, 
evaluates  a  supplier  and  certifies  that  the  supplier  is 
in  compliance  with  Federal  regulations  and  CASE 


standards,  then  other  CASE  members  know  that  they 
can  use  the  supplier  without  having  to  perform  their 
own  audit.  CASE  provides  both  auditing  forms  and 
standards  to  its  members.  There  are  currently  six 
CASE  forms,  although  this  number  changes  as  new 
forms  are  added  and  old  forms  are  retired. 

The  Compliance  Auditor  group  is  responsible  for 
ensuring  that  our  partner  airline’s  maintenance 
operations  are  in  compliance  with  Federal  guidelines 
and  with  its  own  standards.  The  Compliance  Auditors 
use  approximately  32  forms. 

Software  Prototype 

We  have  developed  prototype  software  to  support 
both  Compliance  Auditors  and  Vendor  Surveillance 
Analysts.  Both  prototypes  were  developed  for  use  on 
pen  computers  because  the  auditors  wanted 
capability  similar  to  the  clipboards  they  currently  use. 
The  collected  data  are  stored  in  databases  and  can  be 
printed  out  in  standard  report  formats  or  exported  to 
Microsoft  Word.  This  is  a  vast  improvement  over  the 
current  method  of  manual  transcription  of 
handwritten  paper  forms. 

We  developed  an  application  that  contains  four  of  the 
forms  Vendor  Surveillance  Analysts  use  most 
frequently.  Each  form  is  saved  separately  because  a 
vendor  normally  provides  only  one  supply  or  service. 
An  example  is  shown  in  Figure  2a.  1  (next  page).  The 
application  allows  an  inspector  to  identify  whether  a 
vendor  is  in  compliance  and  to  make  a  comment  for 
each  item  on  the  form,  as  shown  in  Figure  2a.2  (next 
page). 
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The  application  also  contains  links  to  the  CASE 
standards  appropriate  to  the  questions  on  the  auditing 
forms.  This  allows  an  auditor  quickly  to  access  the 
standards  for  reference  while  performing  an  audit.  As 
shown  in  Figure  2a.3,  there  is  a  button  next  to  a 
surveillance  item  (“Does  ROV  hold  an  FAA  repair 
station  certificate?”)  that  identifies  the  standard. 
When  an  auditor  pushes  the  button,  the  standard 
appears  in  Windows  Help,  as  shown  in  the  figure. 
Auditors  like  this  capability  because  they  can  read 
the  standard  and  because  they  can  copy  and  paste  it 
into  their  reports.  Whereas  their  reports  previously 
contained  the  auditor’s  recollection  of  the  standard, 
they  now  contain  the  standard’s  exact  wording. 

We  developed  a  similar  application  for  the 
Compliance  Auditors.  Unlike  the  Vendor 
Surveillance  application,  forms  are  saved  in 
“sessions”;  all  forms  used  in  a  given  audit  are  saved 
together.  This  difference  in  design  results  from  the 
fact  that  a  given  maintenance  facility  of  our  partner 
airline  normally  performs  several  different  types  of 


maintenance  and  requires  multiple  forms.  Because 
the  content  of  the  forms  is  proprietary  to  our  partner 
airline,  we  cannot  publish  examples.  However,  the 
format  and  content  are  very  similar  to  the  Vendor 
Surveillance  forms.  Because  our  partner  airline  has 
proprietary  standards  for  evaluating  their  practices, 
its  managers  have  been  unwilling  to  share  them  with 
us  so  we  could  put  them  on-line. 

Evaluation 

Both  prototypes  are  currently  under  evaluation  at  the 
airline.  We  provided  both  groups  of  auditors  with  a 
number  of  pen  computers  and  copies  of  the  prototype 
software.  Auditors  are  also  using  the  software  on 
their  desktop  computers.  We  expect  the  evaluation  to 
run  sixty  to  ninety  days.  Upon  successful  completion 
of  the  evaluation,  we  plan  to  work  with  the  airline 
and  the  CASE  organization  to  determine  how  these 
concepts  can  be  applied  within  the  broader  aviation 
community. 


CASE  -  [CASE  Air  Carrier  Section  -  Component  Repair/Qvcrhaul  Vendor  (Part  A)] 
Form  Window  Table  of  Contents  General  Comment  Help _ _ 


File"  eM;  _ 


Contents 


J.  Any  license  or  certific ation  required  by  Federal  Aviation 

Regulations  of  any  individual,  equipment,  or  facility  shall  be  kept 
current  and  shall  be  available  for  inspection.  [145.39,  145.103, 

145.17] 

K.  Vendors  that  deal  in  non- aircraft  parts,  materials,  or  maintenanc e 
activities  shall  segregate  the  aircraft  function  from  other  functions 
to  preclude  getting  unapproved  parts  or  materials  on  an  aircraft 
unit.  [145.35] 

L  The  vendor  shall  display  his  repair  station  certificate  and 

operation  specifications  at  a  place  in  the  repair  station  that  is 
normally  accessible  to  the  public  and  is  not  obscured.  1145.19] 

M  The  U.S.  domestic  repair  stations  shall  have  an  active,  FAA 
approved  anti- drug  testing  plan  that  complies  with  FAR  121, 

Appendix  I  The  plan  may  be  the  vendor's  plan,  a  consortium  plan 
to  which  the  vendor  subscribes,  or  an  air  carrier  customer's  plan. 

The  vendor  shall  provide  proof  of  membership  in  the  plan  and 
FAA  acceptance  of  the  plan.  Letters  of  acceptance  or  exemption 

niust.bft  from  the  Dnig  Abatement  Branch  nfhce.nf  Ayiation  . 

A.  Does  hold  aFi  FAA  lepak  certificate?  ^  O  H/A 


p  Coiiment 


111 


Li] 


System  for  Training  of 
Aviation  Regulations 

Terry  Chandler,  Ph,D. 

Galaxy  Scientific  Corporation 


The  ability  to  use  FAA  regulatory  documents  is  a 
requirement  for  all  who  are  associated  with  op¬ 
erations,  maintenance,  and  surveillance  of  aircraft  and 
associated  air  transportation  systems  and  services. 
Schools,  airlines,  manufacturers,  and  the  government 
require  thorough  knowledge,  as  well  as  reasonable 
appreciation,  of  the  Federal  Aviation  Regulations 
(FARs)  and  the  host  of  associated  documents. 

Studying  FAA  regulatory  documents  is  difficult.  In¬ 
structors  are  given  the  arduous  task  of  conveying  the 
meaning  of  subtle  and  seemingly  ambiguous  material 
to  a  student  body  who  do  not  always  recognize  the 
importance  of  what  they  are  learning.  The  two  most 
difficult  aspects  of  learning  the  regulations  are  a) 
learning  how  to  navigate  through  the  FARs  and  other 
related  documents  and  b)  comprehending  the  meaning 
of  particular  statements  within  the  FARs.  FARs  are 
legal  documents  written  precisely  to  define  the  regu¬ 
lations  pertaining  to  aviation.  Unfortunately,  it  is  not 
easy  for  most  people  to  extract  the  intent  of  each 
statement  from  this  style  of  writing.  In  addition,  it  is 
not  always  obvious  where  one  needs  to  look  to  get  a 
complete  sense  of  the  regulations’  intent.  Often,  in¬ 
formation  relevant  to  a  task  is  distributed  across  many 
parts  of  the  FARs.  For  example,  knowing  one's  eli¬ 
gibility  to  perform  an  IFR  inspection  may  not  be  ob¬ 
vious  when  specifications  for  how  to  do  the 
inspection  are  outlined  in  Part  43,  Appendices  E  and 
F,  but  the  privileges  and  limitations  for  who  can  per¬ 
form  the  inspection  are  stated  in  91 .41  Ib  and 
91.413c. 

The  purpose  of  the  System  for  Training  in  Aviation 
Regulations  (STAR)  project  is  to  aid  instructors  in 
teaching  about  the  FARs  (and  other  related  docu¬ 
ments)  by  providing  a  system  that  motivates  the  stu¬ 
dent  to  understand  why  learning  the  FARs  is  both 
relevant  and  necessary,  develops  students'  study  and 
cognitive  skills  in  document  research  and  understand¬ 
ing,  and  c)  makes  the  content  of  the  FARs  more  inter¬ 
esting  and  therefore  more  memorable. 


Our  approach  to  designing  and  developing  STAR  is 
to  incorporate  multimedia  presentations  and  storytel¬ 
ling  techniques  within  several  different  types  of 
learning  environments.  The  goal  is  to  provide  a  com¬ 
prehensive  curriculum  for  acquiring  the  skills  and 
content  necessary  for  efficient  document  research  and 
comprehension. 


The  project  began  in  earnest  on  October  3,  1994. 

In  the  six  months  ending  April  1,  1995,  the  proj¬ 
ect  team  will  have  conducted  a  needs  analysis,  devel¬ 
oped  a  research  approach  guiding  the  design  of 
STAR,  and  built  the  initial  prototype.  A  preliminary 
evaluation  of  the  prototype  will  be  conducted  prior  to 
April  1 .  A  great  deal  of  time  has  also  been  spent  as¬ 
sessing  the  best  way  to  integrate  digital  document 
products  with  government-owned  multimedia  training 
systems.  A  detailed  discussion  of  each  of  these  areas 
is  presented  below. 


We  are  employing  a  user-centered  approach  to 
technical  design  (Chandler,  1994;  Rasmussen, 
1992;  Greenbaum  &  Kyng,  1991;  Norman,  1986). 
Instructors  from  the  FAA  Academy  in  Oklahoma 
City,  three  Part  147  schools,  and  one  flight  training 
academy  were  interviewed  regarding  current  instruc¬ 
tional  practices.  Table  3.1  shows  the  sources  of  in¬ 
formation  for  our  needs  assessment. 


Table  3.1  Sources  of  l.nformation'for.Needs  ■ 


•  Mike  Monroney  Aeronautical  Center 

•  Embry-Riddle  Aeronautical  University 

•  Clayton  State  College  -  Aviation  Dept. 

•  Atlanta  Area  Technical  School 
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Instructors  were  asked  to  identify  the  major  issues 
preventing  students  from  learning  aviation  regulations 
and  to  try  to  envision  how  a  CBT  system  could  ad¬ 
dress  some  of  these  difficult  instructional  issues.  The 
responses  to  our  inquires  were  as  varied  as  the  people 
in  attendance,  but  a  pattern  did  emerge.  Table  3.2 
summarizes  the  learning  issues  instructors  identified 
and  areas  where  CBT  could  support  instruction. 

As  a  result  of  these  interviews,  several  general  re¬ 
search  questions  emerged  to  guide  the  development 
of  STAR  and  its  evaluation.  Table  3.3  lists  the  re¬ 
search  questions.  Our  answer  to  the  question  “How 
do  we  induce  students  to  think  deeply  about  the  sub¬ 
ject?”  will  embody  our  philosophical  approach  to  in¬ 
struction.  This  will  become  more  apparent  during  the 
discussion  below  of  the  design  overview.  “Which 
learning  situations  are  most  effective  for  what  types 
of  learning?”  is  the  question  that  will  guide  the  ex¬ 
periments  for  evaluating  STAR’S  success  as  an  in¬ 
structional  system.  The  other  three  questions  identify 
technical  issues  pertinent  to  user  interface  design  and 
system  functionality  that  we  will  need  to  address 
throughout  the  project. 


Chapter  3 


We  decided  to  focus  our  attention  on  the  training  of 
Aviation  Maintenance  Technicians  (AMTs)  for  the 
first  two  phases  of  this  project  and,  then  to  incorpo¬ 
rate  training  for  pilots  later.  We  sought  the  assistance 
of  Jack  Moore,  Dean  of  Clayton  State  College  - 
Aviation  Department,  as  our  domain  expert  for  this 
phase  of  the  project.  He  and  other  instructors  of  Part 
147  schools  in  Atlanta  have  provided  stories,  exam¬ 
ples,  strategies,  technical  information  and  documen¬ 
tation  to  be  used  as  a  basis  for  developing  the 
curriculum.  We  will  expand  this  information  base  to 
other  Part  147  schools  around  the  country  during  the 
second  phase  of  the  project. 


When  teaching  subtle  information  such  as  avia¬ 
tion  regulations,  there  are  advantages  to  pro¬ 
viding  students  with  many  vantage  points  to  the  same 
body  of  information.  Experiencing  complex  material 
repeatedly  under  different  circumstances  provides  the 
learner  with  multiple  opportunities  to  gain  a  deep  un¬ 
derstanding  of  the  subject.  Each  vantage  point  not 
only  covers  different  aspects  of  the  same  material,  but 
also  reinforces  different  kinds  of  study  skills.  In  addi¬ 
tion,  information  conveyed  through  one  learning  en- 


Students  need  help  in 

©  knowing  who  the  players  are  (e.g.,  owner,  AMT,  pilot,  FAA  maintenance  inspector),  what  their  re¬ 
sponsibilities  are  to  each  other,  and  for  what  regulations  each  must  be  responsible 
®  understanding  the  objectives  of  the  FARs  and  when  and  how  to  apply  them 
®  understanding  the  codependency  of  regulations  to  each  other 
o  learning  to  extract  the  root  meaning  from  the  FARs’  legalese 
®  performing  document  research  procedures 

o  recognizing  when  appropriate  (or  optimal)  procedures  are  applicable 
®  integrating  the  individual  pieces  of  their  job  tasks  into  a  total  picture 

CBT  could  support  instruction  with 

•  a  system  that  supports  multimedia  presentations  during  class  lectures 
©  a  series  of  scenarios  that  elucidate  the  subtle  applications  of  the  regulation 
®  drill  and  practice  sessions  that  show  each  student  where  his  or  her  weak  points  are 
®  a  mechanism  that  allows  Instructors  to  monitor  how  the  students  are  doing 
®  technical  aids  that  support  students  while  they  go  through  the  learning  process 


e  How  do  we  induce  the  students  to  think  deeply  about  the  subject? 

®  Which  learning  situations  are  most  effective  for  what  kinds  of  learning? 

®  When  is  it  more  effective  to  use  what  kinds  of  presentation  types  to  convey  the  salient  points  in  the 
learning  environment? 

e  What  kinds  of  information  retrieval  mechanisms  are  the  most  valuable  to  students?  to  instructors? 

®  How  can  we  translate  digitized  material  meant  for  a  personal  computer  into  a  medium  suitable  for 
distance  learning  broadcasting? 
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vironment  may  be  more  salient  to  a  learner  than  an¬ 
other  approach.  Students  with  different  learning  styles 
are  more  likely  to  benefit  when  different  vantage 
points  are  provided.  In  this  way,  we  provide  students 
not  only  with  multiple  ways  of  viewing  the  informa¬ 
tion,  but  also  with  multiple  opportunities  to  learn. 

The  core  of  the  system  is  a  document  browser  that  has 
full  text  searching  capabilities  both  within  and  among 
documents.  This  allows  students  to  search  and  view 
the  documents  in  their  entirety.  It  also  gives  students 
practice  in  manipulating  the  documents  on-line,  a 
practice  that  we  anticipate  will  be  the  norm  in  the  fu- 


Several  instructors  identified  a  desire  to  have  multi- 
media  clips  punctuate  important  points  they  make 
during  lectures  about  the  regulations.  They  see  this  as 
a  means  for  making  their  instruction  more  interesting 
and  motivational  for  the  students.  Instructors  at  the 
FAA  Academy  in  Oklahoma  are  particularly  inter¬ 
ested  in  this  since  they  are  developing  a  center  for 
distance  learning. 

The  document  browser  is  designed  to  support  effi¬ 
cient  review  of  media  clips  to  augment  class  presen¬ 
tations.  Associated  with  each  document  are  all  the 
multimedia  information  clips  presented  in  the  other 
learning  environments.  For  example,  a  video  about 
instrument  inspection  will  be  indexed  with  the  docu¬ 
ment  section  that  discusses  instrument  inspection. 

The  browser  becomes  an  archive  for  the  documents 
and  all  the  media  clips.  Each  media  clip  is  further  in¬ 
dexed  by  one  of  nine  information  types  listed  in  Table 
3.4.  A  "Very  Important  Point"  information  type,  for 
example,  may  warn  students  of  a  regulation  that  is 
often  violated  and  why  or  how  it  gets  violated.  A  "For 
Your  Information"  information  type  may  point  out  the 
subtle  difference  between  when  an  inspection  must  be 


completed  every  2  years  vs.  every  24  months.  A  "For 
Example"  may  show  a  student  what  a  correct  log  en¬ 
try  looks  like.  By  using  the  documents  themselves  as 
indexes,  augmented  with  classifying  the  media  clips 
into  information  types,  we  have  developed  a  simple 
system  for  organizing  what  is  often  a  very  difficult 
body  of  information  to  catalog.  We  see  this  as  a  natu¬ 
ral  way  for  instructors  to  review  media  clips  relevant 
to  the  material  they  will  be  covering  in  class. 

Surrounding  the  document  browser  (Figure  3.1,  next 
page)  are  four  categories  of  learning  environments: 
overviews,  scenarios,  brain  teasers,  and  technical 
support.  Overviews  show  students  how  FARs  are  or¬ 
ganized,  how  different  parts  are  related  to  each  other, 
and  who  is  responsible  for  what  aspects  of  those 
regulations.  Scenarios  are  interactive  stories  that  set 
each  student  into  a  true-to-life  situation  where  the 
regulations  are  often  subtle.  The  scenarios  present 
students  with  choices  they  need  to  make  within  the 
context  of  a  given  situation  and  show  the  students  the 
consequences  of  those  actions.  It  is  important  to  note 
that  there  is  often  more  than  one  right  or  wrong  an¬ 
swer  and  that  understanding  why  one  action  is  wrong 
in  a  particular  context  is  just  as  important  as  under¬ 
standing  why  another  action  is  right. 

Brain  teasers  present  challenges  to  the  student.  They 
require  students  to  exercise  certain  skills  they  will 
need  to  develop  in  order  to  efficiently  search  the 
regulations  and  understand  what  they  find.  Brain 
teasers  can  vary  in  complexity.  They  can  be  of  the 
"FAR  Jeopardy"  variety  where  students  can  practice 
quick  responses  to  specific  facts.  Brain  teasers  can 
also  be  of  the  "project"  variety  where  solving  a  chal¬ 
lenge  entails  a  deep  understanding  of  both  the  search 
process  and  the  regulations  themselves.  We  see  this 
area  as  a  space  where  instructors  can  develop  their 
own  challenges  for  their  own  students. 


General  Procedures 
Strategies  for  Within  Document  Search 
Strategies  for  Between  Document  Search 
For  Your  Information  (FYl) 

Very  Important  Point  (VIP) 

For  Example 
Personal  Experience 
System  Information 
Terminology 
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Technical  supports  arc  comprehension  aids  such  as  a 
technical  dictionary.  Another  example  is  an  interac¬ 
tive  timeline  showing  the  progression  of  ownership  of 
a  particulai*  type  certificate  by  different  manufactur¬ 
ers.  These  aids  provide  "as  needed  information"  that 
can  be  explored  in  their  own  right  or  use  in  conjunc¬ 
tion  with  other,  more  formal  learning  environments. 

Each  learning  environment  could  be  a  stand-alone 
application.  Together  they  provide  multiple  vantage 
points  for  the  student  to  explore  aviation  regulations. 
Part  of  our  assessment  of  the  total  project  will  be  to 
identify  which  learning  environments  are  most  effec¬ 
tive  for  what  types  of  learning.  By  focusing  on  the 
evaluation  in  this  manner,  we  not  only  will  assess  the 
effectiveness  of  the  application,  but  gain  a  better  un¬ 
derstanding  of  what  types  of  learning  is  occurring  (or 
needs  to  occur)  and  how  we  should  tailor  our  training 
systems  to  achieve  specific  learning  objectives. 

Our  long-term  goal  is  to  develop  authoring  tools  for 
the  most  successful  learning  environments  so  that  the 
domain  expert,  i.e.,  the  instructor,  can  contribute  di¬ 
rectly  to  the  system  rather  than  remain  dependent  on 
application  engineers  for  knowledge  acquisition  and 
implementation.  In  this  way,  the  system  can  take  on  a 
life  of  its  own  becoming  a  repository  of  pedagogical 
expertise  in  aviation  training. 


Digital  documentation  is  a  critical  component  of 
STAR  and  other  document-oriented  training 
systems  such  as  The  Human  Factors  Guide  (see 
chapter  4)  and  The  Inspector  Handbook  (see  chapter 
4),  currently  under  development  at  Galaxy  Scientific  - 
Atlanta.  Over  the  last  four  months,  the  digital  docu¬ 
ments  group  has  identified  what  functionality  such  a 
system  must  support,  who  the  key  commercial  pub¬ 
lishers  are,  and  the  feasibility  for  a  commercial  ven¬ 
dor’s  product  to  be  integrated  into  a  government- 
owned  multimedia  training  system. 

The  details  of  this  evaluation  are  presented  in  chapter 
4.  To  summarize  our  findings,  it  became  apparent  that 
what  is  needed  are  functions  that  give  each  system 
designer  the  power  to  do  full  text  search  of  docu¬ 
ments  and,  the  flexibility  to  display  the  retrieved 
document  in  a  manner  consistent  with  the  training 
system’s  interface.  Though  the  group  continues  to 
evaluate  the  commercial  market,  the  FAA  Hyperme¬ 
dia  Information  System  (HIS)  seems  to  be  best  suited 
for  providing  that  flexibility.  We  have  begun  the 
process  of  extracting  the  functional  components  from 
HIS  so  that  they  can  be  used  by  the  different  training 
systems. 
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For  the  first  phase  of  system  development,  we  be¬ 
gan  building  a  prototype  for  the  document 
browser  and  the  scenario  learning  environment.  Sce¬ 
narios  lend  themselves  to  capturing  the  instructional 
information.  When  a  Part  147  instructor  tells  of  a 
typical  situation  where  interpreting  the  regulations  is 
subtle,  personal  experiences,  examples,  "By  the  Way" 
information,  warnings,  document  search  strategies, 
and  general  procedures  naturally  flow  from  the  telling 
of  the  scenario.  This  information  is  not  found  in  text¬ 
books  or  the  regulations  themselves,  but  is  crucial  to 
an  in-depth  understanding  of  the  regulations.  The  in¬ 
terchange  of  stories  is  not  only  the  most  common  way 
that  we  exchange  information,  but  is  considered  the 
optimal  form  for  retention  of  the  information  received 
(Bruner,  1990;  Shank,  1990).  The  document  browser 
serves  primarily  to  organize  the  information  that  is 
being  collected. 

Scenarios  are  essentially  interactive  stories.  Through 
a  slide  show  presentation,  students  are  told  of  an  un¬ 
clear  situation  where  several  actions  are  possible. 
They  are  asked  a  question  about  what  they  should  do 
given  the  situation  and  are  presented  with  several  ac¬ 
tions  that  they  could  take.  Following  is  the  textual 
passage  presented  to  the  user  for  the  opening  scene  of 
the  special  inspections  scenario. 


You  are  a  technician  with  both  A  and  P  ratings.  Dur¬ 
ing  a  100  hr  inspection  on  an  IFR  equipped  C-172, 
you  notice  that  the  altimeter  and  transponder  have  not 
been  tested  and  inspected  in  the  last  24  months.  When 
you  inform  the  owner  that  these  tests  and  inspections 
are  due,  he  asks:  ‘Tf  these  tests  and  inspections  are 
due,  why  didn’t  you  do  them  as  part  of  the  100  hour 
inspection?”  How  do  you  respond  to  this  question? 


Once  a  student  chooses  an  answer,  a  new  scene  in  the 
scenario  is  presented.  The  new  scene  shows  the  con¬ 


sequences  of  the  action  and  the  rationale  for  why  the 
student  should  or  should  not  have  made  that  choice. 
Imbedded  in  each  explanation  are  references  to  rele¬ 
vant  FAR  passages  and  other  supporting  documents 
and  examples.  For  example,  a  student  might  be  shown 
a  sample  of  a  correct  log  entry  for  the  type  of  mainte¬ 
nance  work  he  or  she  did  or  a  comparison  between 
two  passages  from  the  FARs  where  a  distinction 
needs  to  be  made. 

Although  for  each  scenario  there  is  the  "best"  path  to 
take,  our  objective  is  not  to  train  students  to  take  that 
path.  Rather,  to  get  the  most  out  of  the  scenario,  they 
should  explore  all  the  paths.  By  doing  so,  they  ac¬ 
quire  a  deep  understanding  of  the  situation  and  an  ap¬ 
preciation  for  the  subtle  distinctions  they  need  to 
make  with  respect  to  fully  comprehending  the  intent 
of  the  regulations.  In  this  sense,  there  is  no  right  an¬ 
swer,  only  deeper  understanding.  How  we  entice  stu¬ 
dents  to  explore  all  of  the  scenario  paths  rather  than 
just  to  find  the  "right"  answer  is  part  of  the  larger  re¬ 
search  question  about  inducing  students  to  think 
deeply  about  the  subject. 

While  each  scene  in  the  scenario  has  a  multimedia 
presentation  that  "tells  the  story",  students  also  have 
access  to  other  relevant  material  that  has  bearing  on 
the  situation.  In  the  gray  scale  background  graphic 
used  to  set  the  scene  seen  in  Figure  3.2  (next  page), 
there  are  colored  items  in  the  picture.  When  a  user 
clicks  on  one  of  the  colored  items,  a  video  or  detailed 
graphic  or  explanation  of  the  item  is  presented.  In  our 
instrument  flight  scenario,  for  instance,  clicking  on 
the  altimeter  will  bring  up  a  video  that  explains  the 
functionality  of  an  altimeter  in  the  aircraft.  Also, 
along  the  bottom  of  the  screen  are  buttons  that  access 
other  related  information  categorized  by  information 
type,  e.g.,  FYI,  Personal  Experience,  General  Proce¬ 
dures,  etc.  Students  may  navigate  through  the  sce¬ 
nario  but  also  can  explore  the  details  of  each  scene  in 
its  own  right. 
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As  stated  previously,  the  most  important  research 
question  that  we  will  be  addressing  in  this  project  is, 
"How  do  we  induce  the  students  to  think  deeply  about 
the  subject?"  The  cognitive  and  educational  literature 
claims  that  to  achieve  this  goal  the  student  needs  to 
be  actively  involved  in  the  learning  task  (Brown, 

1992;  Scardamalia  &  Bereiter,  1992;  Resnick,  1991; 
Bransford  et.  al.,  1990;  Papert,  1980).  They  need  to 
be  asking  the  hard  questions  and  trying  to  answer 
them.  There  is  always  a  risk  of  losing  the  students  by 
challenging  them  with  something  that  is  beyond  their 
technical  knowledge,  skill  level,  imagination,  or,  on 
the  opposite  end  of  the  scale,  boring  them  to  death. 
While  scenarios  in  their  present  "canned"  state  do  not 
necessarily  induce  the  students  to  think  for  them¬ 
selves,  they  may  serve  as  a  stepping  stone  to  the  more 
open-ended  challenges  presented  in  the  brain  teaser 
learning  environment.  Scenarios  do  show  the  students 
the  kind  of  thinking  process  they  need  to  employ  in 
order  to  make  sophisticated  decisions  about  ill- 
specified  problems.  By  mimicking  the  reasoning  pre¬ 


sented  in  the  scenarios,  students  should  be  able  to 
solve  the  brain  teaser  challenges.  It  will  be  important, 
when  developing  the  brain  teaser  learning  environ¬ 
ment  in  the  next  phase  of  research,  that  some  of  the 
brain  teasers  are  similar  in  structure  to  those  in  the 
scenarios  so  that  students  can  practice  transferring 
reasoning  skills  to  new  situations. 


The  culminating  event  for  this  phase  of  the  project 
is  to  present  the  STAR  prototype  at  the  34th  An¬ 
nual  Conference  of  ATEC  in  April  1995.  The  confer¬ 
ence  will  provide  a  wide  audience  of  aviation  instruc¬ 
tors  from  across  the  nation.  We  will  use  this  forum  as 
a  vehicle  to  give  us  feedback  on  the  STAR  concept 
and  design,  and  also  an  opportunity  to  tap  conference 
attendees  expertise.  We  will  set-up  several  vehicles 
(including  a  video  camera)  for  capturing  their  stories 
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and  experiences  for  further  development  of  the  sys¬ 
tem. 

In  preparation  for  the  conference,  the  project  team 
will  first  conduct  an  in-house  technical  evaluation  at 
Galaxy  Scientific.  That  session  will  focus  primarily 
on  compatibility  issues  in  the  user  interface  design 
(Maddox  &  Johnson,  1986).  The  instructors  and  a 
select  group  of  students  at  Clayton  State  College  will 
also  have  an  opportunity  to  evaluate  the  STAR  proto¬ 
type.  We  will  ask  them  to  focus  on  system  under- 
standability,  content  accuracy,  information 
presentation  and  ease  of  use  (Maddox  &  Johnson, 
1986).  Formal  evaluations  of  the  system  in  a  class¬ 
room  setting  will  begin  in  Phase  VL 


Phase  V  will  draw  to  a  close  in  April  1995. 

Table  3.5  outlines  the  tasks  for  Phases  VI  and 
VII.  System  Evaluation  will  be  an  important  part  of 
Phase  VI.  We  will  be  analyzing  what  the  students 
learn  from  the  system  in  both  a  non-directed  and  a  di¬ 
rected  setting.  First,  we  will  evaluate  the  robustness 
of  the  system  and  how  students  explore  the  system 
when  it  is  not  tied  to  a  formal  class  activity.  A  history 
trace  will  be  kept  of  each  student’s  activity  on  the 
system.  The  second  part  of  the  evaluation  will  be  in  a 
more  formal  classroom  setting  where  students  will  be 
asked  to  use  the  system  in  the  context  of  one  or  more 
classroom  tasks.  The  focus  here  will  be  on  what  the 
students  learn.  Pre-  and  post-testing  will  be  one  in¬ 
strument  for  this  analysis.  Another  instrument  will  be 
based  on  the  pedagogical  dimensions  developed  by 
Reeves  (1994)  for  evaluating  interactive  learning  en¬ 
vironments.  Analysis  of  students’  history  trace  will 
also  be  made  to  see  if  patterns  emerge  between 
learning  success  and  application  use.  These  results 
will  be  the  bases  for  making  decisions  with  regard  to 
incorporating  intelligent  tutoring  agents  into  STAR. 
In  preparation  for  the  extensive  evaluation  of  the 


system,  the  scenario  and  document  browser  will  be 
developed  into  fully  functional  learning  environ¬ 
ments.  The  major  task  to  fulfill  this  goal  is  producing 
the  curriculum  and  multimedia  materials  to  build  at 
least  one  complete  instructional  unit.  An  example  unit 
could  be  a  series  of  scenarios  about  AMT’s  privileges 
and  limitations.  To  show  the  extent  of  the  instruc¬ 
tional  possibilities,  we  will  also  create  several  differ¬ 
ent  types  of  scenarios  that  are  not  part  of  the  core 
unit.  In  tandem  with  these  other  efforts,  prototypes  for 
the  “overview”,  “technical  support”  and  “brain 
teaser”  learning  environments  will  be  developed  and 
initial  evaluations  of  their  interface  design,  robust¬ 
ness,  and  content  accuracy  will  be  conducted  during 
Phase  VI. 

A  comparative  study  between  traditional  instruction 
and  instruction  incorporating  STAR  as  an  integral 
part  of  the  curriculum  will  be  made  during  Phase  VII. 
In  preparation  for  this  study,  the  overview,  technical 
support,  and  brain  teaser  prototypes  will  be  devel¬ 
oped  into  full  learning  environments.  The  content  of 
the  training  system  will  be  expanded  to  training  pilots 
and  the  potential  for  converting  the  training  systems 
into  authoring  systems  will  be  assessed. 


The  STAR  project  gives  us  an  opportunity  to  bring 
out  the  complexity,  subtlety,  and  interesting  as¬ 
pects  of  what  is  normally  thought  to  be  a  dry  subject. 
It  provides  a  vehicle  for  practicing  skills  in  document 
research  and  complex  decision-making.  It  gives  stu¬ 
dents  practice  with  computerized  tasks  that  they  will 
be  expected  to  use  with  facility  in  the  near  future.  It 
provides  a  vehicle  for  interacting  with  the  subject 
matter  from  several  different  vantage  points,  increas¬ 
ing  the  chances  of  each  student  acquiring  an  in-depth 
understanding  of  the  material.  And,  as  researchers,  it 
gives  us  the  opportunity  to  evaluate  what  instructional 
vehicles  are  best  suited  to  achieve  the  learning  objec- 


Phase  VI 

®  Convert  the  scenario  and  document  browser  into  fully  functioning  Learning  Environments. 

•  System  evaluation  -  non-directed  setting. 

•  System  evaluation  -  forma!  classroom  setting. 

®  Develop  prototypes  of  the  overview,  technical  support  and  brain  teaser  learning  environments. _ 

Phase  VII 

•  Convert  the  overview,  technical  support  and  brain  teaser  into  fully  functional  Learning  Environments. 
®  Conduct  comparative  study  between  traditional  instruction  and  instruction  incorporating  STAR. 

©  Expand  content  of  system  to  include  curriculum  for  Aviation  Flight  Schools. 

©  Assess  potential  for  converting  training  systems  into  authoring  systems. 
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Digital  documentation  systems  are  a  key 

component  of  the  Human  Factors  in  Aviation 
Maintenance  research  program.  This  study  of  digital 
documentation  systems  was  undertaken  in  an  effort  to 
address  problems  associated  with  the  publication, 
distribution,  and  use  of  large  quantities  of  printed 
information  in  the  aviation  industry.  Digital 
documentation  systems  have  an  advantage  over  paper 
or  microfiche  documents  in  terms  of  compactness  of 
information.  For  example,  a  bookshelf  of  manuals 
and  reference  materials  can  be  stored  electronically 
on  a  single  CD-ROM.  Other  advantages  of  electronic 
documents  include  the  potential  cost  savings  and 
faster,  more  effective  access  to  needed  information. 
With  a  paper/microfiche  system,  a  maintenance 
technician  could  spend  considerable  time  researching 
information  for  a  given  maintenance  task  on  an 
aircraft.  With  a  properly  developed  digital 
documentation  system,  the  time  can  be  substantially 
reduced,  perhaps  to  only  a  few  hours.  Air  carriers  will 
save  money  from  quicker  turn-around  times  on 
maintenance  tasks.  General  Aviation  will  benefit  from 
reduced  paper-based  research  associated  with  Annual 
Inspections. 

The  conversion  from  printed  to  electronic 
information,  however,  is  not  without  costs,  and  the 
research  program  is  investigating  ways  of  efficiently 
creating,  accessing,  and  maintaining  digital 
documentation  with  a  focus  on  ensuring  an  interface 


that  is  compatible  with  the  aviation  users.  The 
Hypermedia  Information  System  (HIS)  has  been 
developed  to  investigate  digital  documentation 
storage  and  retrieval  issues.  Hypermedia  is  a 
computer-based  technology  that  allows  non-linear 
access  to  information.  The  information  may  be  in  the 
form  of  text,  graphics,  audio,  video,  or  animation.  For 
more  information  on  the  HIS  system,  see  Chapter  6  of 
the  Phase  IV  report  (FAA/AAM  &  GSC,  1994). 

This  chapter  describes  research  and  development 
activities  related  to  digital  documentation  completed 
in  the  past  year.  Section  4. 1  details  the  process  for 
converting  documentation  from  paper  to  electronic 
form.  Section  4.2  describes  how  the  initial  prototype 
of  the  digital  Human  Factors  Guide  was  designed 
and  developed.  Section  4.3  describes  the  contents  of 
CD-ROM  #3.  Finally,  Section  4.4  discusses  future 
plans  for  digital  documentation  research. 


The  process  of  converting  a  document  into  digital 
form  requires  several  steps.  Figure  4.1  illustrates 
the  basic  digital  documentation  process.  This  section 
describes  basic  steps  used  to  process  a  paper 
document  for  the  HIS:  convert  it  to  digital  form,  add 
markups,  index  the  text,  and  structure  the  topics. 
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4.1.1  Coevert  to  Digital  Form 

If  no  electronic  version  of  the  document  is  available, 
the  first  step  is  to  convert  printed  text  to  digital  form. 
For  small  documents,  it  may  be  feasible  to  type  the 
document  using  a  word  processor;  for  larger 
documents,  typing  may  be  too  labor-intensive. 
Fortunately,  commercially  available  hardware  and 
software  semi-automates  this  process.  A  scanner  is 
similar  to  a  photocopier;  it  is  attached  to  a  personal 
computer.  Optical  Character  Recognition  (OCR) 
software  converts  a  scanned  image  of  text  into  an 
ASCII  text  file,  i.e.,  OCR  software  “recognizes” 
bitmap  characters  and  “types”  the  corresponding 
ASCII  character  into  a  text  file.  OCR  software  does 
not  preserve  formatting  such  as  bolding  or  italics.  For 
more  information  on  the  OCR  process  and  a  review 
of  commercial  OCR  software  products,  see 
Mantelman,  1994. 

Since  neither  typing  nor  OCR  conversion  is  error- 
free,  a  major  part  of  this  step  is  to  verify  the  output 
for  accuracy.  Verification  can  also  be  time- 
consuming  and  tedious,  although  standard  word 
processing  tools  like  spell  checkers  can  assist.  Some 
other  techniques  have  been  developed  to  locate  errors 
quickly.  For  example,  the  same  document  may  be 
processed  by  two  typists,  or  by  two  OCR  packages. 
Resulting  files  are  compared  using  a  software  utility 
program  that  locates  any  differences  between  the  two 
files.  Since  differences  often  correspond  to  errors, 
this  technique  helps  automate  the  verification  process. 

Since  many  documents  contain  figures  and  images,  as 
well  as  text,  the  conversion  to  digital  form  is  not 
complete  until  non-text  portions  of  the  document  are 
processed.  Scanners  can  also  assist  in  this  process. 
Depending  on  the  quality  of  the  original  paper 
document  and  the  capabilities  of  the  scanner,  varying 
amounts  of  post-scanning  cleanup  may  be  necessary 
to  obtain  good  quality  graphics.  In  instances  where 
the  item  does  not  scan  well,  it  may  be  necessary  to 
recreate  the  graphic  or  figure  using  a  software 
drawing  package. 

It  is  difficult  to  offer  a  general  rule  for  how  long  it 
takes  to  complete  this  first  step.  The  necessary  time 


depends  on  several  factors,  including:  the  document’s 
quality  and  length,  the  number  and  complexity  of 
graphics,  and  speed  and  capabilities  of  personnel, 
tools,  and  techniques.  A  simple  document  with  few 
graphics  can  be  processed  relatively  quickly,  but  a 
large  document  with  special  layout  can  take 
substantial  time.  For  example,  the  Air  Transportation 
Operations  Inspector’s  Handbook  is  approximately 
five  hundred  pages  long,  laid-out  in  columns.  The 
conversion  took  over  three  person- weeks  to  complete. 

Given  the  labor  intensive  nature  of  conversion,  it  is 
extremely  beneficial  to  omit  this  step.  This  is  possible 
only  when  an  electronic  version  of  the  original 
document  exists.  However,  even  when  an  electronic 
copy  exists,  some  processing  may  be  needed  to  have 
electronic  data  in  a  format  compatible  with  HIS  tools 
running  on  IBM  PC-compatible  computers.  For 
example,  if  the  digital  document  exists  on  a 
mainframe,  the  data  would  need  to  be  converted  to  an 
IBM  PC-compatible  text  file  format. 

4.1.2  Add  Markups 

As  soon  as  an  electronic  version  of  a  document  is 
available,  the  next  step  is  to  add  special  markups  to 
the  file.  Markups  are  standardized  sequences  of 
characters  used  to  “mark”  portions  of  the  text  with 
formatting  and  hypermedia  information.  Figure  4.2 
(next  page)  shows  Galaxy  Markup  Language  (GML) 
syntax  for  some  common  markups.  GML  was 
developed  a  few  years  ago  for  the  HIS  system  and  is 
similar  to  standard  markup  languages  like  SGML 
(Standard  General  Markup  Language)  and  HTML 
(Hypertext  Markup  Language). 

HIS  allows  for  three  methods  of  completing  the 
markup  step:  use  the  point  and  click  authoring  mode 
in  the  HIS  viewer,  write  and  use  a  macro,  or  write 
and  use  a  filter  program.  Each  method  is  described 
below.  The  markup  method  chosen  depends  on  the 
size  of  the  document,  the  number  of  markups  to  be 
made,  the  format  of  the  electronic  file,  and  the 
programming  capabilities  of  the  person  doing  the 
processing. 
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Common  Formatting  Markups 

Bold:  <B>  ...fexf...  </B> 

Italics:  </I> 

Underline:  ...text...  <fU> 

F ont:  <F” (font), (fontsize)” >  ...text. 

Indent:  <indent”(level),(pre- indent  string” >  ...text. ..  </indent> 

C enter :  <c enter> ...text...  </c enter> 

Flush  Right  <right> . . .  text . .  <f  right> 

Margins  <margin”(left  margin),  (right  margin)  > ..  .text .. 

Common  Hypermedia  Markups 


Tag: 

Hot  Table: 

Hot  Graphics: 
HotMe^a: 

Hot  Executable: 
Hot  Link: 


<rG”(tag  reference),(caption)”> 
<HT''(file),(caption)”> ...text..  </HT> 
<HG”(file),(caption)”> ...fexf...  ^G> 
<HM”(file),(caption),(start),(enC> ...text..  </HM> 
<IE”(command  line) ”>  ...fexf...  </HE> 

<HL”(tag reference)’'> ...text. .  </HG> 


4. 1.2.1  Use  HIS  Author  Mode 

A  person  with  no  programming  skills  can  use  the  HIS 
viewer’s  authoring  mode  for  adding  markups  to  a 
document.  Author  mode  allows  a  text  file  to  be 
loaded  into  the  viewer  and  marked  up  manually. 
Manual  markups  are  accomplished  by  a  user  selecting 
portions  of  the  text  and  then  choosing  the  type  of 
markup  desired,  e.g.,  bold,  topic,  or  hotword. 

For  example,  if  a  user  wants  to  create  a  hotword 
linking  to  a  graphics  file,  he  or  she  would  select  the 
portion  of  the  text  he  or  she  wants  to  be  the  hotword, 
and  then  select  the  menu  option  to  create  the  link.  As 
shown  in  Figure  4.3  (next  page),  a  dialog  box  is  then 
displayed  that  allows  the  user  to  specify  the  type  of 
link  to  be  created.  The  authoring  system  interprets  the 
user’s  point  and  click  actions  as  instructions  to  add 
the  proper  markup  to  the  text  file.  At  the  end  of  each 
authoring  session,  the  user  must  save  changes  to  save 
markups  that  were  added.  While  this  method  is 
feasible  for  small  documents  with  few  markups,  it  is 
too  tedious  and  time-consuming  for  large  documents 
with  a  substantial  number  of  markups. 

4. 1.2.2  Write  and  Use  a  Macro 

The  process  of  adding  markups  can  be  automated 
with  the  help  of  macro  facilities  in  some  word 
processing  packages.  For  example,  Microsoft  Word 


contains  a  macro  facility  which  records  a  series  of 
mouse  and  keyboard  actions  in  a  Word  Basic 
program.  A  user  needs  only  minimal  programming 
skills  to  edit  these  macro  programs.  Such  commercial 
tools  can  be  used  to  convert  formatting  information  in 
Word  files  to  corresponding  GML  markups  and  to 
add  other  GML  markups  such  as  topic  tags  and 
hotword  links. 

One  of  the  greatest  benefits  of  such  automation  is  that 
an  unlimited  number  of  files  can  be  processed  once 
the  macro  is  written  and  tested.  If  the  contents  of  a 
document  change  over  time,  a  filter  automating  the 
markup  process  saves  time  and  money  by  keeping  the 
on-line  system  current  with  changes.  If  the  documents 
to  be  processed  are  Word  files  (or  a  format  easily 
converted  to  Word),  this  method  is  the  obvious 
choice  for  adding  markups. 

4. 1.2. 3  Write  and  Use  a  Filter  Program 

Writing  a  filter  program  to  add  markups  to  a  file 
requires  the  most  programming  skill.  Before  the 
program  can  be  written,  one  must  analyze  the 
document  to  see  how  it  is  organized,  i.e.,  Volumes, 
Parts,  Chapters,  Sections,  etc.  A  user  can  then  write  a 
filter  that  uses  lexical  tools  automatically  to  place 
markups  in  the  appropriate  places.  Once  the  filter  is 
written,  it  can  be  tested  on  a  representative  file  to 
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locate  and  fix  any  mistakes.  If  the  document  is  fairly 
uniform,  writing  and  debugging  a  filter  does  not  take 
very  long.  However,  the  filter  for  PARS  took 
approximately  a  week  to  write  because  PARS  are  not 
uniform,  i.e.,  SPARS  and  appendices  are 
intermingled  with  Parts. 

After  the  filter  is  debugged,  a  user  can  write  a  batch 
file  to  run  the  filter  on  all  of  the  document’s  files. 
Depending  on  the  document’s  size  and  the  number  of 
markups  to  be  added,  run-time  may  take  from  3  to  20 
minutes  per  document.  Although  filter  programs  arc 
useful  for  automating  the  bulk  of  the  mark-up 
process,  it  is  likely  that  some  markups  will  need  to  be 
added  manually.  A  user  can  add  these  additional 
markups  directly  to  the  GML  file  with  a  text  editor; 
the  HIS  Authoring  mode  can  also  be  used  to  add  a 
small  number  of  mark-ups. 

4.1.3  Index 

The  third  step  in  the  process  is  to  index  marked-up 
files.  Indexing  is  a  technical  term  for  building  a 


database  to  support  full-text  searching  and 
hypermedia  linking.  Por  full-text  searching,  the 
database  stores  every  word  in  the  document  and  its 
location  in  the  document.  Certain  words  are  not 
indexed  because  no  one  would  want  to  search  for 
them;  these  “stop”  words  include  articles  (e.g.,  a,  an, 
the)  prepositions  (e.g.,  of,  at,  in)  and  pronouns  (e.g., 
she,  he,  it,  you). 

Por  hypermedia  linking,  the  database  stores 
information  for  two  primary  types  of  markups:  tags 
and  hotwords.  The  tag  markup  designates  topics  for 
the  Table  of  Contents.  The  database  stores  the 
location  of  each  tag  markup  so  the  user  can  jump 
directly  from  the  topic  in  the  Table  of  Contents  to  the 
associated  text.  The  hotword  markup  designates 
words  or  phrases  in  the  document  which  link  to  other 
information.  The  database  stores  the  location  of  each 
hotword  and  the  location  of  its  associated  text, 
graphic,  video,  or  audio. 
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HIS  tools  include  an  indexing  program  that  processes 
GML  files.  For  a  single  small  document,  indexing 
may  take  only  a  few  minutes;  for  large  documents,  it 
can  take  several  hours.  The  HIS  indexing  tool  allows 
a  developer  to  index  a  group  of  files  as  a  batch  job. 
The  developer  can  set  up  the  job  and  allow  it  to  run 
unmonitored  overnight.  This  feature  minimizes  the 
impact  of  a  slow  indexing  process.  This  process  can 
be  repeated  over  several  nights  to  index  very  large 
documents.  For  example,  it  took  about  eighteen  hours 
to  index  the  FAR  text  into  an  HIS  database. 

4.1.4  Structure  Topics 

In  the  HIS  system,  topics  correspond  to  items  listed  in 
the  Table  of  Contents,  such  as  the  chapter,  section, 
and  subsection  headings.  In  the  markup  step,  all 
topics  are  identified  with  the  tag  markup.  The  indexer 
stores  each  topic’s  location  in  the  database,  so  a  user 
can  jump  from  the  Table  of  Contents  to  any  topic’s 
beginning.  The  final  step  in  the  conversion  process  is 
to  structure  topics  into  an  outline  so  the  HIS  Table  of 
Contents  viewer  displays  the  topics  hierarchically. 

To  illustrate  the  effect  of  the  structuring  process, 
Figure  4.4  shows  HIS  displaying  the  Table  of 
Contents  for  an  example  document,  both  before  and 
after  structuring.  For  the  iinstnictiired  document, 
notice  that  all  topics  are  listed  without  any  indenting. 
After  the  topics  are  structured,  HIS  displays  only 
topics  at  the  highest  level  of  the  outline,  such  as  the 
chapter  titles.  When  the  user  clicks  a  page  icon,  the 
next  outline  level  appears. 

The  structuring  process  does  not  require  a  lot  of  time, 
compared  with  the  time  required  for  other  steps  in  the 
process.  This  step  is  partially  automated,  so  a  small 


program  must  be  written  to  add  level  information  to 
topics  in  the  HIS  database.  A  structuring  program  is 
customized  to  the  syntax  of  the  topics  in  a  document; 
therefore,  it  will  only  be  valid  for  documents  with  the 
same  syntax.  For  small  documents,  run-time  can  take 
less  than  an  hour;  for  larger  documents  such  as  the 
PARS,  run-time  may  take  several  hours. 

4.1.5  Discussion 

The  digital  documentation  process  obviously  requires 
some  investment  of  time.  The  actual  time  required 
depends  on  several  factors,  including  the  size  and 
state  of  the  original  document.  To  illustrate  all  the 
steps  for  the  HIS  system,  in  this  section,  we  discussed 
the  four  basic  steps  necessary  if  a  large  document 
does  not  exist  in  digital  form.  There  are  substantial 
time  savings  to  be  gained  if  the  process  can  start  with 
an  electronic,  rather  than  a  paper,  document. 

We  did  not  discuss  additional  steps  required  if  audio, 
video  and/or  animation  are  to  be  included  in  the 
digital  documentation.  Additional  time  and  effort  is 
required  to  locate  and/or  create  such  media,  as  well  as 
to  process  it  into  a  form  the  HIS  system  can  use.  If 
the  additional  media  already  exists,  and  is  easily 
located,  costs  are  lower  than  if  original  media  must  be 
created.  Appropriate  footage  may  not  exist,  or  may 
take  a  long  time  to  locate.  When  appropriate  footage 
is  located,  copyright  permissions  must  be  obtained 
before  it  can  be  used  in  the  project. 

The  benefits  of  digital  documentation,  with  or  without 
additional  media,  must  be  weighed  against  the  costs 
for  converting  and  maintaining  on-line 
documentation.  Informal  evaluations  of  the  HIS 
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system  have  been  conducted,  with  positive  results. 
The  benefits  of  quicker  and  more  accurate  access  to 
information,  as  well  as  portability  of  electronic  data, 
provide  sufficient  benefits  to  warrant  conversion  of  a 
variety  of  aviation  maintenance  data  to  digital  form. 


One  of  the  major  digital  documentation  projects 
completed  during  the  past  year  was  the  design 
and  development  of  a  prototype  Electronic  Human 
Factors  Guide.  This  Electronic  Guide  (E-Guide)  is 
the  digital  counterpart  of  the  paper-based  Human 
Factors  Guide  for  Aviation  Maintenance  (the  Guide). 
The  Guide  describes  fundamental  human  factors 
concepts  and  guidelines  for  aviation  maintenance 
supervisors  and  technicians.  Its  goal  is  to  provide 
practical,  usable  guidance  to  supervisors  and  planners 
in  the  aviation  maintenance  industry. 

The  E-Guide  utilizes  the  HIS  functionality  to  improve 
access  to  the  Guide's  content.  It  provides  the  HIS 
full-text  search  capability,  as  well  as  hypertext  linking 
between  chapters.  The  E-Guide  expands  on  the 
Guide's  content  by  incorporating  video  that 
supplements  the  paper-based  Guide's  text  and  still 
images. 

The  HIS  authoring  tools  were  selected  for 
development  of  the  E-Guide  over  commercially 
available  tools  for  three  primary  reasons.  First,  the 
HIS  technology  met  the  functional  requirements  that 
were  desired.  Second,  most  commercially  available 
tools  that  meet  the  functional  requirements  do  not 
meet  the  cost  requirements.  That  is,  substantial  fees 
are  required  for  distributing  the  commercial  software 
used  to  view  the  electronic  information,  typically 
around  $50/copy.  Documents  developed  with  HIS 
authoring  tools  do  not  incur  any  “per  copy”  costs. 
Finally,  customization  is  possible  using  the  internally 
developed  HIS  software.  If  a  new  feature  is  needed  or 
a  change  in  an  existing  function  is  required,  the  HIS 
authoring  tools  can  be  modified.  Such  control  is  not 
possible  with  commercial  software  tools. 

In  this  section,  we  describe  design  issues  and 
interface  features  of  the  prototype  system.  We 
conclude  with  a  summary  of  initial  user  feedback 
about  the  E-Guide  and  the  modifications  we  are 
implementing. 


4.2.1  Designing  the  Electronic  Guide 

The  E-Guide  was  designed  in  coordination  with  the 
paper-based  Guide.  As  with  the  paper-based  Guide, 
there  were  three  design  goals  for  the  E-Guide: 

•  it  should  be  readily  accessible  to  the 
aviation  community 
®  it  should  be  easy  to  maintain 
®  it  should  be  easy  to  use. 

In  this  section,  we  discuss  how  we  achieved  these 
goals  during  the  design  and  development  of  the  initial 
E-Guide  prototype. 

4.2. 1.1  Achieving  the  Accessibility  Goal 

One  goal  of  the  Human  Factors  Guide  research 
program  is  to  provide  wide  and  easy  access  to  the 
information  written  for  the  Guide.  The  E-Guide  will 
be  accessible  in  two  ways:  CD-ROM  and  Internet.  A 
CD-ROM  disc  holds  approximately  650  megabytes  of 
data;  this  is  sufficient  space  for  the  Guide's  text  and 
media,  as  well  as  relevant  documentation  such  as  the 
FAA/AAM  meeting  proceedings  and  phase  reports. 
Because  such  a  large  quantity  of  information  can  be 
stored  on  one  CD-ROM  disc,  the  E-Guide  can  easily 
be  distributed  to  the  aviation  community  at  a 
reasonable  cost.  The  cost  to  replicate  each  disk, 
including  packaging  materials,  is  approximately 
$1.65. 

The  research  team  is  investigating  the  Internet  as  an 
alternative  means  for  information  distribution  (see 
Chapter  5,  Skyway).  The  Guide's  complete  text  will 
be  on  the  Internet  to  ensure  wide  distribution  of  the 
information,  especially  to  those  without  a  CD-ROM 
player.  To  date,  one  draft  chapter  of  the  E-Guide  has 
been  successfully  converted  to  HTML  and  placed  on 
the  Internet. 

4.2.1 .2  Achieving  the  Maintenance  Goal 

The  Guide  is  intended  to  provide  practical  guidance 
to  aviation  maintenance  supervisors  and  planners. 
Since  issues  and  problems  of  maintenance  constantly 
change,  the  Guide  needs  periodic  updating  to  address 
new  problems.  The  challenge  is  to  keep  the 
information  in  the  Guide  current  at  minimal  cost. 

The  paper  version  solves  this  problem  by  providing 
the  Guide  in  a  three-ring  binder,  instead  of  in  book 
form.  A  chapter  can  be  added,  eliminated,  or 
upgraded  without  discarding  the  whole  book.  This 
keeps  the  cost  to  upgrade  and  distribute  information 
at  a  minimum. 
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The  cost  to  upgrade  the  system  includes  the  cost  of 
modifying  both  digital  documentation  and  interface 
software,  as  well  as  the  cost  to  redistribute  the 
software.  Redistribution  costs  are  minimized  by  using 
CD-ROM  and  the  Internet.  The  cost  of  modifying 
software  depends  on  the  effort  involved  in 
reprocessing  portions  of  the  digital  documentation. 
We  streamlined  the  HIS  digital  documentation 
process  in  the  following  ways  to  minimize  this  cost: 

•  The  Guide  is  being  developed  in  Word  to 
eliminate  the  need  to  convert  from  paper  to 
digital  form. 

•  We  created  a  customized  Word  macro  to 
automate  markup.  The  macro  automatically 
deletes  unnecessary  formatting  information  from 
the  Word  files,  adds  the  required  hypermedia 
commands,  and  saves  the  file  in  the  proper  HIS 
text  format. 

•  We  created  a  separate  HIS  database  for  each 
chapter.  This  modularizing  of  the  databases 
allows  a  chapter  to  be  added,  deleted,  or 
modified  without  reprocessing  the  contents  of 
other  chapters. 

4.2.1. 3  Achieving  the  Ease  of  Use  Goal 

Both  the  paper  and  electronic  versions  of  the  Guide 
are  designed  to  be  easy  to  use.  The  E-Guide  retains 
ease-of-use  features  of  the  paper  Guide,  including  its 
organizational  structure  of  the  sections  and  the 
chapter  icons.  There  are  other  factors  to  be 
considered  in  designing  and  implementing  a  useable 
software  interface  that  go  beyond  the  features 
inherited  from  the  paper  version. 

User  interface  design  is  a  critical  project  element 
because  it  plays  such  a  major  role  in  users’ 


acceptance  of  the  electronic  version.  A  user, 
especially  a  computer  novice,  is  more  likely  to  use  the 
E-Guide  if  the  interface  allows  him  or  her  to  focus  on 
finding  and  using  the  Guide's  information,  rather  than 
focusing  on  navigating  and  using  the  software.  The 
research  team  developed  a  customized  interface  for 
the  E-Guide  which  exploits  the  Human  Factors 
Guide's  specific  structure,  rather  than  simply  using 
the  Hypermedia  Information  System’s  (HIS’s) 
generic  interface. 

To  ensure  an  intuitive,  user-friendly  program  for  the 
custom  interface,  we  are  using  the  cyclic  design 
model  to  design  and  develop  the  E-Guide.  Figure  4.5 
shows  the  four  iterative  steps  involved  in  the  process: 
analyze,  design,  implement,  test.  We  have  completed 
one  cycle  to  date. 

To  further  ensure  a  usable,  commercial  appearance 
for  the  E-Guide,  the  researchers  evaluated  interface 
features  of  twelve  commercial  CD-ROM  applications. 
Each  application  was  evaluated  for  its  ease  of 
navigation,  overall  ease  of  use,  screen  layout,  and 
media  integration.  For  details  of  this  evaluation,  see 
Hartzell,  1994.  The  E-Guide  prototype  design  was 
based  on  this  evaluation,  as  well  as  human  interface 
design  research  findings  and  guidelines. 

4.2.2  The  Interface  Features 

In  this  section,  we  describe  interface  features  of  the 
E-Guide’ s  initial  prototype.  We  follow  this  section 
with  a  summary  of  initial  user  evaluation  feedback 
and  a  description  of  the  resulting  modifications  we 
will  make  to  the  initial  prototype  software. 

4.2.2. 1  The  Introduction 

The  E-Guide’ s  introduction  is  a  real  “attention- 
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getter.”  It  starts  animation  of  the  title:  Human  Factors 
Guide  for  Aviation  Maintenance.  A  video  clip 
introducing  the  FAA/AAM  research  program  follows 
the  animation.  This  introduction  plays  until  a  user 
presses  any  key  or  clicks  a  mouse  button;  the  system 
proceeds  to  display  the  Table  of  Contents. 

4.2.2.2  The  Table  of  Contents 

Tlie  Table  of  Contents  in  the  paper  Guide  is  in  the  form 
of  a  conventional  text  outline  of  chapter  titles.  The  E~ 
Guide  presents  the  Table  of  Contents  as  a  unified  scene 
(Figure  4.6.  Since  the  Guide  is  intended  for  members 
of  the  aviation  community,  we  chose  a  hangar  for  the 
scene.  Each  graphical  image  in  the  hangar  represents  a 
chapter  in  the  Guide.  We  chose  each  image  to  illustrate 
the  chapter  it  represents,  while  always  maintaining  the 
aviation  maintenance  theme.  For  example,  a  time  clock 
with  punch  cards  represents  the  chapter  on  Shiftwork 
Scheduling.  This  pictorial  Table  of  Contents  serves  as 
an  overview  map  from  which  the  user  can  access  any 
chapter.  Pointing  at  an  image  with  the  cursor  displays  a 
pop-up  displaying  the  chapter’s  title;  selecting  the 
image  displays  the  chapter’s  Introduction  in  the 
Information  Viewer. 


4.2.2.3  The  Information  Viewer 

The  Information  Viewer  displays  the  Gitide^s  content 
(Figure  4.7,  next  page).  The  Information  Viewer’s 
design  is  critical  for  meeting  the  ease-of-use  goal;  this 
is  the  primary  screen  for  accessing  information  in  the 
Human  Factors  Guide.  We  conducted  an  analysis  of 
user  needs  to  identify  displays  and  controls  to  include 
in  the  viewer.  We  designed  the  Information  Viewer  to 
use  dedicated  locations  for  all  display  areas  and 
controls:  all  information  and  program  functionality  is 
visible  on  the  screen.  In  this  section,  we  describe  key 
features  of  the  Information  Viewer:  the  Section  buttons, 
the  Text  Window,  the  Media  Window,  and  the  E- 
Guide  Control  Buttons. 

4. 2. 2.4  Section  Buttons 

Each  of  the  Guide's  chapters  is  divided  into  twelve 
sections:  Introduction,  Background,  Issues  and 
Problems,  Regulatory^  Requirements,  Concepts, 
Methods,  Reader  Tasks,  Guidelines,  Related  Issues, 
Where  to  Get  Help,  References,  and  Further  Reading. 
In  the  E-Guide,  sections  are  represented  by  twelve 
section  buttons  grouped  together  just  above  the  Text 
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Window  (which  displays  the  section’s  text).  Each  of 
the  twelve  section  buttons  has  a  distinct  icon.  The 
icons  are  metaphors  for  familiar  objects;  this  allows 
users  to  have  quicker  recognition  of  each  section 
button.  If  a  user  is  unsure  what  an  icon  represents,  the 
section’s  name  is  displayed  in  a  help  balloon  near  the 
button  whenever  the  user  places  the  cursor  on  top  of 
the  button. 

A  user  selects  a  section  button  to  view  a  different 
section  of  the  current  chapter.  When  the  user  selects  a 
section  button,  the  button  is  inverted,  and  the  mouse 
cursor  changes  to  an  hourglass  until  the  Information 
Viewer  has  retrieved  the  section  text.  This  design 
gives  users  immediate  access  to  information  in  any 
section  and  allows  them  quickly  to  identify  what  text  is 
currently  displayed  by  noting  which  section  button  is 
currently  inverted. 

4. 2. 2. 5  Text  Window 

As  mentioned  above,  the  Text  Window  is  located 
below  the  section  buttons.  This  window  displays  the 
selected  section’s  text  in  the  same  format  as  the 
paper-based  Guide.  The  text’s  size  is  slightly  larger 


than  the  paper  version’s  to  make  it  easier  to  read  the 
computer  screen. 

Within  the  text,  some  words  are  displayed  in  a 
different  color;  such  words  are  called  hotwords,  A 
hotword  indicates  that  there  is  associated  text  or 
media  related  to  that  word.  The  association  is  called  a 
hyperlint,  it  provides  a  software  connection  between 
the  hotword  and  another  document,  graphic  image,  or 
definition.  Hotwords  give  users  rapid  access  to 
information;  selecting  the  hotword  displays  its 
associated  text  or  media.  Section  text  is  displayed  on 
the  left  side  of  the  screen;  graphics  and  other  media  are 
displayed  in  the  Media  Window  on  the  right.  A  user 
can  view  text  and  its  corresponding  media 
simultaneously. 

4. 2. 2. 6  Media  Window 

The  Media  Window  displays  tables,  figures,  video,  and 
animation  associated  with  the  current  chapter’s  content. 
In  the  Information  Viewer,  the  Media  Window  is 
located  to  the  right  of  the  Text  Window.  Below  the 
Media  Window,  the  Media  Description  box  contains  a 
short  description  of  the  image  currently  displayed  in  the 
Media  Window.  Until  a  user  selects  a  figure  or  media 
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file,  the  Media  Window  displays  the  FAA  AAM  logo. 
The  logo  serves  as  a  filler,  blending  the  Media  Window 
into  the  background  and  preventing  the  user  from  being 
distracted  by  an  empty  window. 

The  Media  Control  Buttons  are  directly  beneath  the 
description.  The  graphic  on  each  button  illustrates  the 
media  the  button  controls,  e.g.,  a  video  camera  for  the 
video  control,  a  camera  for  the  photo/figure  control, 
and  a  chart  for  the  table  and  charts  control.  When  the 
current  chapter  has  no  media  of  a  given  type,  the 
corresponding  control  button  is  disabled.  A  user  may 
select  an  enabled  media  button  to  display  a  list  of 
associated  media  for  the  current  chapter.  For  example, 
when  a  user  clicks  on  the  video  control  button,  a  list  of 
video  clips  relevant  to  the  current  chapter  is  displayed. 
A  user  can  select  any  item  in  the  list  to  view  the 
associated  video.  When  a  user  selects  a  figure,  table,  or 
other  media  file,  the  Media  Window  replaces  its 
previous  contents  with  the  newly  selected  file.  The 
transition  effect  draws  the  user's  focus  to  the  Media 
Window. 

The  Media  Window’s  default  size  is  a  relatively  small 
180  X  130  pixels.  This  size  is  appropriate  for  video 
clips  or  animation  playback;  however,  a  table  or  a 
figure  is  typically  much  larger.  The  Media  Window 
displays  a  scaled-down  version  of  tables  and  figures  in 
overview.  To  see  the  image’s  details,  the  user  can 
enlarge  the  table  or  figure  to  its  original  size.  The 
enlarged  table  or  figure  is  displayed  in  a  separate 
window  with  the  caption  as  the  window’s  title.  The 
main  Information  Viewer  window  is  deactivated  while 
this  enlarged  window  is  displayed,  preventing  the  user 


from  getting  lost  or  confused  by  there  being  too  many 
windows  on  the  screen. 

Tables  and  figures  in  the  E-Guide  are  taken  directly 
from  the  paper  Guide.  The  graphics  are  stored  as  image 
files,  preserving  their  format  and  color.  The  audio, 
video,  and  animation  media,  which  are  not  part  of  the 
paper  Guide,  had  to  be  collected  and  processed  for  the 
E-Guide.  The  current  design  of  the  Information  Viewer 
allows  the  following  file  formats;  WAV  files  for  audio, 
AVI  files  for  video,  FLl  and  FLC  files  for  animation, 
GIF  and  BMP  for  still  images. 

4.2. 2.7  Electronic  Guide  Control  Buttons 

E-Guide  control  buttons  access  navigational  and 
system  functions.  These  control  buttons  are  located  at 
the  bottom  of  the  Information  Viewer  screen.  The  basic 
functions  of  the  buttons  are  as  follows: 

©  Next  and  Previous  chapter  buttons 
display  the  next  or  previous  chapter  in 
the  Text  Window 

©  Table  of  Contents  button  displays  the 
Table  of  Contents  overview  map 
©  Go  To  button  allows  a  user  to  go  directly 
to  any  section  of  any  chapter 
©  Search  button  allows  a  user  to  search  the 
Guide  for  specific  words  or  phrases 
©  Print  button  allows  a  user  to  print 

selected  text  or  graphics  from  the  Guide 
®  Help  button  displays  the  on-line  E-Guide 
Help  window 

®  Exit  button  exits  the  E-Guide. 
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Many  of  these  functions  are  straightforward.  A  user 
simply  clicks  the  appropriate  control  button  and  its 
corresponding  action  occurs.  Some  functions  require 
additional  input,  typically  supplied  in  a  dialog  box.  For 
example,  Figure  4.8  shows  the  “Go  To”  dialog  box  in 
which  a  user  must  give  the  desired  chapter  and  section. 
“Search”  is  one  of  the  E-Guide’ s  most  useful  functions; 
it  requires  additional  user  input.  This  function  is  used 
much  as  one  might  use  a  combination  of  the  Table  of 
Contents  and  the  Index  in  the  paper-based  Guide.  If  a 
user  wants  information  on  a  specific  topic  in  the  paper- 
based  Guide,  he  or  she  might  scan  headings  in  the 
Table  of  Contents  or  look  up  the  specific  topic  in  the 
Index. 

In  the  E-Guide,  a  user  selects  “Search”  to  locate 
relevant  material.  A  dialog  box  helps  a  user  provide 
information  necessary  for  the  search  (see  Figure  4.9, 
next  page)  with  options  to  search  the  current  section, 
the  current  chapter,  or  the  whole  book.  A  user  must 
specify  one  or  more  words  or  phrases.  To  search  for  a 
single  word  or  a  phrase,  a  user  types  the  desired  term  or 
phase  in  the  “Find”  box  and  selects  the  Search  button. 

When  a  user  has  supplied  necessary  information,  he  or 
she  executes  the  search  by  clicking  on  the  Search 
button.  The  hourglass  cursor  is  displayed  until  the 
search  is  complete.  A  dialog  box  then  displays  a  list  of 
chapter  numbers  and  sectibn  names  in  which  the  term  is 


found.  As  shown  in  Figure  4.9  (next  page),  the  system 
automatically  highlights  a  search  term  contained  in  the 
currently  displayed  section. 

The  E-Guide  is  also  capable  of  complex  searches  with 
wildcards.  A  wildcard  search  means  that  a  user  can  use 
wildcard  characters  to  search  for  variations  of  a  word. 
The  E-Guide  supports  two  standard  wildcard 
characters:  “?”represents  any  single  character,  and 
represents  one  or  more  characters.  For  example,  a 
search  for  “circ*”  would  find  terms  such  as  “circa,” 
“circadian,”  “circular,”  and  “circumstances.”  A  search 
for  “circ?”  would  yield  only  “circa”  from  the  above  list. 

4.2.3  User  Feedback  and  Interface  Modifications 

We  demonstrated  the  first  prototype  of  the  E-Guide  at 
the  Ninth  FAA  AAM  Meeting  on  Human  Factors  in 
Aircraft  Maintenance  and  Inspection.  In  addition, 
several  attendees  used  the  prototype  in  a  workgroup 
setting,  identifying  several  interface  and  usability 
issues.  The  issues,  notes,  and  “wish-list  features”  are 
summarized  below,  along  with  the  modifications  we 
will  make  to  the  E-Guide: 

•  Text  Display:  An  attendee  suggested 

implementing  an  option  to  display  the  text  in  a 
full-screen  window.  Although  while  in  the  full¬ 
screen  mode,  the  user  cannot  view  the  supporting 
media  simultaneously.  There  may  be  times  when 
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The  reality  of  the  fatigue-induced  performance  problems 
associated  with  shiftwork  has  been  evident  for  many  years. 
However,  only  recently  have  the  underlying  mechanisms  that 
cause  these  effects  been  identified  and  explained.  The  key 
concepts  related  to  shiftwork-related  performance  problems  are 
described  below. 

Rhythms 

The  term  circadian  refers  to  the  fact  that  the  types  of  cycles 
we  described  in  previous  sections  have  a  period  of  nearly,  but 
not  exactly,  a  24-hour  day.  The  word  circa  is  Latin  for  about  or 
approximately.  The  physiological  and  psychological  variables 
that  follow  our  internal  clock  are  said  to  be  circadian.  Body 
temperature  is  an  example  of  a  circadian  cycle.  Figure  4-1 
shows  the  approximate  body  temperature  cycle  for  an 
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the  user  is  only  interested  in  reading  text.  We  will 
implement  this  option. 

®  Table  of  Contents:  The  Table  of  Contents 
represents  each  chapter  in  the  Guide  with  a 
graphic  image.  Although  this  approach  provides  a 
unifying  theme  and  lends  a  commercial  look  to  the 
prototype,  some  users  may  be  more  comfortable 
with  a  traditional  Table  of  Contents.  Participants 
suggested  that  the  E-Guide  include  an  option  to 
switch  between  the  two  Tables  of  Contents.  We 
will  implement  this  option. 

®  Iconized-Section  Buttons:  Due  to  users’ 
unfamiliarity  with  icons  and  contents  of  the 
Human  Factors  Guide,  they  did  not  utilize  the 
section  buttons  very  much.  Participants 
recommended  adding  a  menu  list  of  all  sections  as 
an  option  to  the  section  buttons.  We  will 
implement  a  menu  that  allows  a  user  to  make  a 
selection  with  the  mouse  or  the  keyboard. 

©  Tables  and  Figures:  Since  current  tables  and 
figures  are  image  files,  users  cannot  perform 
searches  on  their  information.  Users  identified 
expanding  the  search  capability  to  include  this 
information  as  a  necessary  modification:  important 
information  resides  in  tables  and  figures.  We  will 
investigate  the  feasibility  of  adding  such  a  feature. 


»  Hyperlinks:  At  the  time  of  the  conference,  we  had 
not  implemented  linking  from  one  portion  of  the 
text  to  another.  Participants  indicated  their  desire 
to  have  footnotes  linked  to  the  associated 
reference.  They  were  also  interested  in  links 
among  the  E-Guide  and  other  FAA  and  DOT 
documents  referenced  in  the  text.  We  will 
implement  hyperlinks  to  references;  we  will 
implement  linking  to  additional  documents  as  lime 
and  money  permit. 

Other  general  feedback  participants  gave  us  on  both  the 
paper-based  and  electronic  versions  of  the  Human 
Factors  Guide  included  the  following: 

o  Glossary:  Attendees  commented  that  many 
aviation  maintenance  managers  may  not  be 
familiar  with  the  technical  meaning  of  terms  (e.g., 
fatigue)  we  use  in  the  Guide.  Some  attendees 
suggested  including  definitions  from  an  aviation 
dictionary.  We  plan  to  add  a  glossary  to  both 
versions  of  the  Guide, 

©  Examples:  The  attendees  recommended  adding  a 
section  in  the  Guide  of ‘'Examples  of  Best/Current 
Practices”  from  the  airline  industry.  We  will 
include  two  new  sections  in  both  versions  of  the 
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Guide:  Example  Scenarios  and 
Acknowledgments. 


4.3  FAA/MM  CDbRQM;#3„„ , 


or  the  third  consecutive  year,  one  of  the  digital 
documentation  task’s  major  deliverables  is  a  CD- 
ROM.  As  in  the  past,  the  current  CD-ROM  contains 
several  software  programs  produced  as  part  of  the 
FAA  AAM  Human  Factors  in  Aviation  Maintenance 
research  program  (Figure  4.10).  In  this  section,  we 
briefly  describe  the  contents  of  CD-ROM  #3.  Readers 
may  find  additional  details  on  a  particular  application 
by  referring  to  the  corresponding  chapter  in  this 
report. 

4.3.1  Hypermedia  Information  System(HIS) 

The  Hypermedia  Information  System  (HIS)  project 
provided  the  impetus  for  developing  the  first  CD- 
ROM.  During  the  past  year,  we  have  improved  and 
expanded  the  HIS’  features  and  contents.  The  1995 
version  of  HIS  provides  over  5,000  pages  of 
information  related  to  aviation  maintenance  and 
inspection,  including  the  following:  Human  Factors  in 
Aviation  Maintenance  Phase  Reports  and  Meeting 
Proceedings,  Federal  Aviation  Regulations  (Parts  1- 


200),  the  Airworthiness  Inspector’s  Handbook  (Order 
8300.10),  and  the  Air  Transportation  Operations 
Inspector’s  Handbook  (Order  8400.10). 

The  HIS  program  contains  a  graphical  user  interface 
that  makes  it  easy  for  a  user  to  browse  through  these 
documents,  and  hypermedia  technology  affords  rapid 
access  to  specific  information.  The  full-text  search 
function  allows  searching  within  and  across  all 
documents  in  the  system.  Storing  digital 
documentation  electronically  on  CD-ROM  is  one 
feasible  method  for  improving  distribution  and  access 
to  information. 

4.3.2  Electronic  Human  Factors  Guide 

Since  the  paper-based  Human  Factors  Guide  will  not 
be  published  until  later  this  year,  CD-ROM  #3 
contains  only  a  demonstration  version  of  the 
Electronic  Human  Factors  Guide  that  is  similar  to  the 
initial  prototype  described  in  this  chapter.  However, 
since  the  text  for  all  chapters  is  under  revision,  only 
two  revised  Chapters  are  included  in  the 
demonstration  program:  Chapter  1  (Human  Factors) 
and  Chapter  4  (Shiftwork  and  Scheduling). 
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4.3.3  Ergonomics  Audit  Program-ERNAP 

The  ERgoNomic  Audit  Program  (ERNAP)  is  a 
computerized  job  aid  that  helps  managers  evaluate  or 
design  ergonomically  efficient  procedures  and 
systems  for  maintenance  or  inspection.  ERNAP  is 
simple  to  use;  it  evaluates  existing  and  proposed  tasks 
and  setups  by  applying  ergonomic  principles.  If  an 
evaluation  is  unfavorable,  ERNAP  suggests 
ergonomic  interventions. 

The  complete  ERNAP  system  contains  twenty-three 
modules  spanning  Pre-Maintenance,  Maintenance, 
and  Post-Maintenance.  The  CD  contains  an  initial 
prototype  of  the  software.  A  complete  version  is  to  be 
published  on  the  Electronic  Human  Factors  Guide 
CD-ROM  in  June  1995. 

4.3.4  Coordinating  Agency  for  Supplier 
Evaluation  (CASE) 

The  vendor  audit  program  for  the  Coordinating 
Agency  for  Supplier  Evaluation  (CASE)  Air  Carrier 
Section  is  an  adaptation  of  the  Aviation  Safety 
Inspector  job-aiding  software.  Auditors  from  each 
participating  airline  perform  inspections  of  their 
respective  vendors  and  contribute  their  findings  to 
CASE  resources.  The  software  is  designed  to  help 
auditors  collect  required  data  during  on-site 
inspections  of  vendors. 

The  fully  functional  CASE  program  is  designed  to 
operate  on  a  pen  computer  running  Microsoft 
Windows  for  Pen  Computing.  The  CD-ROM  contains 
a  demonstration  program  illustrating  the  main 
features  without  requiring  the  special  operating 
system. 

4.3.5  Office  of  Aviatioin  MedIcBine  Video  Brochure 

The  Office  of  Aviation  Medicine  Video  Brochure 
describes  the  FAA’s  Office  of  Aviation  Medicine 
(AAM)  goals,  organization,  and  work  in  a  series  of 
short  video  clips.  The  software  is  designed  to  be  used 
either  on  a  "public  access"  computer  (video  kiosk)  or 
on  a  personal  computer.  The  AAM  Video  Brochure 
uses  the  Microsoft  Video  for  Windows  system,  which 
displays  digital  video  on  a  computer  without 
requiring  special  hardware. 

4.3.6  PENS  Video  Brochure 

The  PENS  Video  Brochure  describes  the 
Performance  Enhancement  System  (PENS)  research 
program  in  a  series  of  short  video  clips.  The  Video 
Brochure  software  is  designed  to  be  used  either  on  a 


“public  access”  computer  (video  kiosk)  or  on  a 
personal  computer.  The  PENS  Video  Brochure 
displays  digital  video  on  the  computer  without 
requiring  special  computer  hardware. 

PENS  is  an  electronic  performance  support  system 
designed  for  Aviation  Safety  Inspectors.  It  provides 
data  entry  and  validation  support,  as  well  as  on-line 
access  to  policy  guidance  such  as  Federal  Aviation 
Regulations,  Airworthiness  Directives,  and 
Inspector’s  Handbooks.  The  system  is  currently  used 
by  the  FAA  Flight  Standards  Service. 


Some  current  digital  documentation  research  and 
development  efforts  continue  through  the  next 
year.  We  will  continue  work  on  the  Electronic  Human 
Factors  Guide.  The  first  complete  E-Guide  will  be 
published  on  CD-ROM  in  June  1995.  As  we  revise 
the  paper-based  Human  Factors  Guide,  the  E-Guide 
will  also  be  updated. 

Work  on  the  HIS  system  continues.  As  our  work  the 
E-Guide  demonstrated,  there  are  specialized  needs  for 
digital  documentation,  i.e.,  a  generic  interface  like  the 
HIS  may  not  always  be  desirable.  However,  a  custom 
interface  may  well  want  pieces  of  the  HIS’ 
functionality.  We  now  have  the  idea  of  carving 
modules  out  of  the  HIS  software  for  use  in  other 
programs.  We  used  this  process  for  the  Search 
function  used  in  the  E-Guide.  We  are  likely  to 
continue  modularization  of  the  HIS  during  the 
coming  year.  We  will  publish  a  new  HIS  on  CD- 
ROM  #4  in  March  1996.  This  CD  will  also  contain 
software  developed  for  other  projects  within  the 
overall  research  program. 

We  have  new  research  and  development  avenues  to 
address  in  the  coming  year.  Current  systems  have 
demonstrated  the  feasibility  of  digital  documentation 
for  the  aviation  industry,  but  technological  and 
organizational  changes  have  occurred  since  we  began 
our  research.  New  hypermedia  and  multimedia 
development  tools  are  available.  Commercial  systems 
providing  large-scale  imaging  tools  for  document 
management  have  been  developed.  New  digital 
documentation  standards  are  evolving  as  commercial 
companies  enter  the  market  with  products  providing 
aviation-specific  digital  documentation  libraries.  Our 
research  and  development  work  should  not  replicate 
services  now  available  commercially. 
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Our  future  research  will  adapt  to  the  aviation 
maintenance  industry’s  current  needs.  We  have  to 
pose  questions  as  to  what  needs  commercial  suppliers 
are  already  meeting  (or  will  be  meeting  in  the  near 
future)  and  what  needs  remain  for  further  research 
and  development.  In  conjunction  with  this  type  of 
needs  analysis,  we  need  to  review  new  tools, 
standards,  and  techniques  formally.  We  can  then 
define  further  investigations  to  match  technology  and 
needs. 
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5.0  INTRODUCTION  - 


The  Office  of  Aviation  Medicine  (AAM)  Human 
Factors  in  Aviation  Maintenance  research  team 
has  been  exploring  alternative  methods  for  dissemi¬ 
nating  the  products  from  the  research  program.  Ex¬ 
amples  include  publication  of  project  results  on  CD- 
ROM,  the  Human  Factors  Guide  for  Aviation  Main- 
tenance,  and  annual  meetings  and  reports.  The  pro¬ 
gram  has  included  efforts  to  involve  the  research  and 
user  communities  in  its  decision-making  processes. 


Another  avenue  for  disseminating  information  is 
through  an  on-line  electronic  information  source. 
This  new  distribution  channel  has  been  termed  the 
FAA  Information  Skyway. 

This  report  presents  our  vision  of  what  the  Skyway 
is,  of  our  progress  with  our  User  Needs  Survey,  a 
survey  of  existing  services,  and  a  snapshot  of  the 
World-Wide  Web  (WWW)-based  Skyway  to  date. 

As  shown  in  Figure  5.1,  the  AAM  will  use  the  Infor- 


E-mall, 
fax,  video- 
conferencing 
with  users  and 
researchers 


User  ^ 
communication 
coming  into  /\AM 


r  Program 
►  newsletter  or 
journal  distribution 

N  / 

Users  requests  / 
for  new  information  / 


Gopher,  FTP  and  WWW  site 
for  new  product 
advertisement  and  info. 


The  FAA 

Information  Skyway 


41 


Information  Skyway 


Chapter  5 


mation  Skyway  to: 

1 .  Disseminate  information  from  the  Human 
Factors  Research  Program,  Office  of  Avia¬ 
tion  Medicine,  and  the  FAA  to  all  Internet 
users 

2.  Maintain  and  update  official  aviation-related 
documents  and  standards  generated  by  the 
Office  of  Aviation  Medicine  for  immediate 
world-wide  use 

3.  Provide  additional  Maintenance  Human 
Factors -oriented  Internet  services,  such  as 
notification  bulletins,  information  archiving 
and  retrieval,  and  conducting  world-wide 
discussion  groups. 

A  substantial  portion  of  the  FAA  Information  Sky¬ 
way  will  be  based  on  the  WWW,  a  Standard  General 
Markup  Language  (SGML)-based  hypermedia  in¬ 
formation  layer  available  through  the  Internet.  The 
WWW  allows  hypertext  access  across  all  WWW 
hosts  and  documentation.  Most  WWW  hosts  are  gov¬ 
ernment-  sponsored  research  organizations  or  com¬ 
mercial  publishers. 

Internet  and  the  WWW  are  explosively  growing  me¬ 
diums  for  information  access  (Stefanac,  1994).  Pre¬ 
viously  restricted  to  government  research, and 
educational  firms,  Internet  recently  opened  access  to 
general  business  organizations.  Seven  thousand  busi¬ 
nesses  and  organizations  now  have  15  million  Inter¬ 
net  users — ^there  are  one  million  more  users  each 
month.  Over  a  recent  12  month  period,  WWW  traffic 
increased  341,634%;  and  a  new  network  is  joining 
the  Internet  every  10  minutes.  Twenty-one  large 
Bulletin  Board  Systems  (BBSs)  have  also  connected 
to  the  Internet,  at  least  for  e-mail  transfer.  More  than 
half  of  all  registered  networks  are  now  commercial. 
Surveys  have  also  been  done  on  existing  WWW  us¬ 
ers  (Pitkow,  1994). 

Immediate  benefits  for  the  AAM  of  the  FAA  Skyway 
include  publicity  and  immediate  distribution  of  the 
Office's  public  information,  research  results,  and  of¬ 
ficial  notifications.  Previous  AAM  experiences  with 
electronic  distribution  of  research  information,  by 
way  of  CD  and  SGML,  technically  position  the  AAM 
to  pursue  this  form  of  publication. 

Long-term  benefits  of  the  FAA  Skyway  are  based  on 
current  research  and  development  activity  among 
commercial  aviation  manufacturers  and  FAA  AAM. 
Commercial  aviation  manufacturers  are  beginning  to 
distribute  documentation  electronically  in  an  SGML 


format.  (Remember  that  WWW  is  SGML-based, 
too.)  Current  AAM  and  FAA  research  projects  are 
evaluating  how  to  use  portable  computers  to  support 
maintenance  and  inspection  activities.  The  merging 
of  portability,  world-wide  access,  and  a  plethora  of 
electronic  aviation-related  documentation  will  serve 
to  bring  timely  information  to  our  maintenance  and 
inspection  users. 


The  Information  Skyway  User  Needs  Survey  has 
been  created  empirically  to  determine  needs  in 
the  community.  The  survey’s  intent  is  to  establish 
what  members  of  the  Aviation/Human  Factors  com¬ 
munity  have,  need,  and  want  from  existing  or  poten¬ 
tial  on-line  electronic  information  services. 
Specifically,  the  survey  includes  questions  on  what 
classes  of  FAA  information  and  services  community 
members  desire,  what  computer  resources  users  have 
access  to,  and  individual  affiliations  and  job  func¬ 
tions.  The  survey  will  be  distributed  to  people  across 
the  airline,  academic,  and  government  sectors.  The 
survey  is  included  in  this  report  as  Chapter  5  -  Ap¬ 
pendix  . 

The  question  arises  as  to  how  innovative  an  approach 
the  Skyway  should  take.  An  innovative  strategy  at¬ 
tempts  to  identify,  refine,  and  specialize  emerging 
technologies  and  prepare  users  for  the  new  and  hope¬ 
fully  ubiquitous  technology.  Alternatively,  a  more 
conservative  and  applied  strategy  minimizes  risk  by 
employing  only  the  most  widely  available  tools,  if 
not  innovative  tools. 

The  Skyway  occupies  the  more  innovative  position 
on  this  scale.  The  Internet  is  a  major  information 
technology  and,  while  not  yet  on  every  desktop,  is 
here  to  stay.  We  predict  that  the  Internet  will  be  a 
primary  source  for  electronic  information  -  including 
Aviation  and  Human  Factors  information. 


The  User  Needs  Survey  will  help  us  determine 
what  the  Skyway  should  do,  what  information  it 
should  include,  and  how  it  should  be  accessed.  There 
are  two  immediately  apparent  ways  for  members  of 
the  public  to  access  computerized  on-line  informa¬ 
tion:  the  Bulletin  Board  System  (BBS)  and  the  Inter¬ 
net. 
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BBSs  are  typically  accessed  with  low-speed  modems 
over  standard  telephone  lines.  A  BBS  is  often  hosted 
on  a  PC  with  many  modem  ports.  One  advantage  of 
BBSs  is  that  they  require  modest  equipment:  a  PC 
with  a  low-speed  modem  and  modest  graphics,  and 
no  pre-established  account.  BBS  services  typically 
include  E-Mail  (amongst  users  of  the  BBS),  real-time 
CHAT  conversations,  and  uploading  and  download¬ 
ing  files.  Usually,  these  systems  do  not  offer  ad¬ 
vanced  services  such  as  document  searching, 
hypertext,  or  multimedia. 

The  Internet  is  a  computer  network  pioneered  in  the 
1960s.  Today,  many  millions  of  users  in  the  public, 
academic  and  governmental  sectors  share  in  this 
global  fabric.  Internet  services  are  typically  more  ad¬ 
vanced  than  a  BBS’s  and  include  E-Mail,  file 
up/down  loading,  hypertext,  multimedia,  video  con¬ 
ferencing,  etc.  Until  recently,  it  was  difficult  to  con¬ 
nect  to  the  Internet.  Only  university  researchers  or 
government  officials  could  afford  the  specialized 
communications  connections  or  could  use  the  UNIX 
environment.  However,  access  is  now  much  easier. 
New  protocols  (such  as  Serial  Line  Internet  Protocol 
or  SLIP),  modems,  public  domain  software  and 
commercial  Internet  Service  Provider  (ISP)  compa¬ 
nies  make  access  feasible  for  many  people.  This 
trend  continues;  in  fact,  reports  are  that  the  upcoming 
Windows  95  will  come  bundled  with  Internet  soft¬ 
ware  and  that  the  Internet  will  reside  on  most  desk- 
tops. 

The  Internet,  specifically  the  World  Wide  Web,  is 
our  first  experiment  in  the  Information  Skyway.  We 
do  not  see  the  Skyway  necessarily  as  a  single  me¬ 
dium  or  service,  so  our  initial  foray  into  an  Internet- 
based  Skyway  does  not  preclude  future  work  with 
BBSs  or  any  other  means  of  effectively  delivering  in¬ 
formation  electronically. 

5.2.1  Internet  Services 

Before  discussing  Internet  services,  we  will  briefly 
discuss  methods  of  access.  Until  recently,  Internet 
connectivity  required  high-speed  digital  communica¬ 
tions  found  only  in  sophisticated  labs  and  large  of¬ 
fices.  With  the  introduction  of  SLIP  protocol  and 
high-speed  modems,  a  typical  PC  can  cost-effectively 
establish  a  true  Internet  connection.  ISPs  offer  a  SLIP 
dial-up  bridge  into  the  Internet  for  a  few  dollars  per 
month.  In  fact,  Internet  access  is  now  as  easy  as  dial¬ 
ing  up  a  bulletin  board. 


We  made  a  survey  of  Internet  Services,  seeking  out 
both  mainstream  and  emerging  Internet  technology. 
Services  we  investigated  included  E-Mail,  Gopher, 
video  conferencing,  Lotus  Notes,  WWW,  File  Trans¬ 
fer  Protocol  (FTP),  ListServers,  and  Multiple  User 
Domains  (MUDs).  We  gave  most  attention  to  WWW 
and  FTP  as  potential  services  due  to  their  widespread 
use,  high  growth,  and  appropriateness  for  digital 
documentation. 

5.2.1 .1  Electronic  Mail 

E-Mail  is  a  core  Internet  service  and  is  available  in 
many  environments  other  than  the  Internet.  Different 
E-Mail  systems  typically  communicate  via  Gateways. 
For  example,  E-Mail  is  routinely  exchanged  between 
CompuServe,  America  On-Line  (AOL),  and  the  In¬ 
ternet  users,  as  well  as  many  localized  proprietary 
LAN-based  E-Mail  systems  such  as  ccMail,  PROFS, 
and  Microsoft  Mail.  Text-based  E-Mail  can  be  en¬ 
hanced  with  multimedia  attachments,  as  well  as  with 
groupware-oriented  enhancements  such  as  ListServ¬ 
ers  (see  Section  5.2. 1.5,  next  page). 

5.2.1. 2  The  World  Wide  Web 

The  WWW,  commonly  referred  to  as  “the  Web”,  is 
one  of  the  fastest  growing  Internet  services.  A  user 
views  WWW  documents  called  “pages”  by  using  a 
WWW  viewer  or  browser.  Many  browser  programs 
are  available  for  most  platforms,  including  NCSA 
Mosaic,  CELLO,  NetCruiser,  and  NETSCAPE.  Web 
pages  may  include  text,  graphics,  or  multimedia. 
Links  within  the  text  allow  the  user  to  branch  off  to 
other  WWW  pages  or  other  Web  sites  anywhere  in 
the  world.  The  ability  to  move  between  documents 
and/or  host  computers  by  using  links  embedded  in 
the  text  is  called  “hypertext”.  WWW  pages  may  also 
be  searched  for  key  words  or  phrases. 

WWW  documents  use  the  HyperText  Markup  Lan¬ 
guage  (HTML)  format  for  providing  text  and  graphi¬ 
cal  hypertext.  The  HTML  format  is  standardized  and 
extensible.  Web  servers  may  provide  back-end  pro¬ 
grams  triggered  by  the  reader’s  manipulation  of  the 
page.  For  example,  a  WWW  page  may  present  an 
interactive  form  or  provide  a  front-end  to  a  large  da¬ 
tabase  system. 

WWW  pages  may  include  references  or  links  to  the 
other  Internet  services.  For  example,  the  user  may 
click  on  a  link  that  triggers  an  FTP  download  of  a 
particular  file  or  that  makes  a  link  to  a  Gopher  menu. 
In  this  way,  WWW  subsumes  many  other  Internet 
services. 
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5.2.1. 3  FTP 

File  Transfer  Protocol  (FTP)  is  perhaps  the  oldest 
Internet-based  service.  Simply  put,  FTP  allows  users 
to  retrieve  files  from  sites  on  the  network.  FTP  ar¬ 
chives  are  maintained  throughout  the  Internet.  FTP 
users  access  files  organized  in  hierarchical  directories 
on  specific  hosts.  There  are  many  topic-specific  FTP 
archives.  For  example,  Microsoft  maintains  an  ar¬ 
chive  for  Visual  Basic  software  and  there  are  FTP 
general  archives  dedicated  to  electronic  versions  of 
popular  manuals. 

5.2.1. 4  Gopher 

Gopher  is  a  precursor  to  WWW  and  presents  infor¬ 
mation  in  a  hierarchical  menu.  Users  view  a  linear 
list  of  items  which  lead  to  other  Gopher  menus  or  to 
text.  Gopher’s  simplicity  allows  it  to  easily  run  on 
almost  any  client  interface,  including  text-based 
terminals.  Like  the  WWW,  Gopher  items  link  easily 
to  other  Gopher  items  on  other  distant  nodes.  Figure 
5.2  shows  one  example  series  of  Gopher  menus. 

5.2. 1.5  ListServers 

One  popular  service  is  the  ListServer  (also  known  as 
a  mail  reflector).  ListServers  are  an  extension  to  E- 
Mail.  ListServers  are  established  for  particular  topics 


(similar  to  UseNet  groups).  Users  send  specific  E- 
Mail  “command”  messages  to  the  server  to  subscribe 
and  unsubscribe  from  the  list  and  to  request  lists  of 
current  subscribers.  Once  subscribed,  users  send 
messages  to  the  group  and,  likewise,  receive  mes¬ 
sages  from  the  entire  group.  Since  a  ListServer  is 
based  on  simple  E-Mail  mechanisms,  any  E-Mail 
user,  on  the  Internet  or  not,  may  utilize  the  service.  A 
potential  Skyway  service  is  one  or  more  ListServers 
for  topics  such  as  “Human  Factors  in  Aviation.” 

5.2.1. 6  Other  Services 

Other,  more  exotic  Internet  services  include  MUDs 
and  Video  Conferencing.  MUDs  are  text-based 
groupware  programs  originally  intended  for  multi¬ 
player  role-playing  games.  MUDs  have  been  sug¬ 
gested  as  a  new  vehicle  for  real-time  conferences 
where  participants  interact  with  each  other  in  ‘rooms’ 
based  on  a  particular  sessions,  topics,  etc. 

While  seamless  video  requires  higher  bandwidth 
links,  several  real-time  video  conferencing  systems 
exist  on  the  Internet.  The  CU„SeeMe  video  confer¬ 
encing  system  is  a  simple,  low-bandwidth  video  sys¬ 
tem  which  has  been  employed  in  K-12  schools.  The 
DRUMS  system  from  Sprint  integrates  Silicon 
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Graphics  Indy  systems,  video  cameras,  and  high¬ 
speed  TCP/IP  (Transmission  Control  Proto¬ 
col/Internet  Protocol)  links  to  bring  together  profes¬ 
sional  studio  video  producers  and  their  clients. 

There  are  other  important  network-based  services 
which  are  not  necessarily  Internet-based  at  all,  but 
still  may  be  accessed  by  the  Internet.  For  example, 
Lotus  Notes  is  a  groupware  product  running  on 
LANs  (Local  Area  Networks)  such  as  Novell.  Corpo¬ 
rations  are  using  Lotus  Notes  for  E-Mail,  group 
scheduling,  group  coordination,  etc. 


The  present  accessible  Skyway  is  a  collection  of 
WWW  documents.  This  implementation  will  be 
added  to  and  changed  as  the  results  of  the  User  Needs 
Survey  are  analyzed.  The  following  sections  of  this 
report  detail  the  status  of  this  WWW  effort.  The  first 
section  considers  how  we  access  the  Internet,  and  the 
following  sections  consider  the  actual  WWW  imple¬ 
mentation. 

5.3.1  Internet  Service  Providers 

When  discussing  services,  it  is  often  important  to 
distinguish  between  providing  the  service  and  con¬ 
suming  the  service.  Computer  terminology  for  this  is 
client  vs.  server.  It  is  typically  easier  to  be  the  client 
of  an  Internet  service  than  to  be  the  server.  For  ex¬ 
ample,  there  are  now  many  popular  and  inexpensive 
packages  in  any  bookstore  that  allow  a  user  to  access 
the  Internet  (and  become  a  client).  For  instance,  it  is 
relatively  easy  to  setup  an  IBM  PC  (or  a  Mac)  to  ac¬ 
cess  the  many  FTP  and  WWW  information  sources 
now  on  the  Internet.  The  Skyway  must  be  a  server 
publishing  WWW  information. 

Several  alternatives  exist  for  the  Skyway  server.  The 
server  is  where  the  Skyway  information  resides  and 
is  where  the  WWW  and  FTP  protocols  are  imple¬ 


mented.  One  approach  is  to  employ  an  Internet 
Service  Provider’s  (ISP)  UNIX  machine  and  a  SLIP 
connection.  The  ISP’s  machine  maintaining  the  ac¬ 
tual  data  storage  is  continuously  connected  to  the  In¬ 
ternet.  Galaxy  Scientific  corporation  connects  to  the 
ISP’s  machine  as  needed  over  a  low-speed  modem 
and  uploads  our  information.  This  method  is  the  most 
cost-effective  for  small  scale  prototyping,  but  offers 
the  least  control  and  poor  cost-per-bit  for  larger  scale 
data  storage.  Another  approach  is  to  establish  an  on¬ 
premises  host  which  provides  all  data  storage  and 
server  implementation.  This  approach  requires  more 
extensive  set-up  and  hardware. 

We  are  now  using  an  off-site  ISP  host.  Specifically, 
an  Atlanta-based  ISP  named  MindSpring,  Inc.,  pro¬ 
vides  us  with  disk  storage,  FTP,  and  WWW  server 
access,  and  a  SLIP  account  for  approximately 
$50/month  plus  $1 /Mbytes/Month  storage  fee. 

We  have  investigated  establishing  an  on-site  host. 
Some  cost  estimates  for  doing  so  are  shown  in  Table 
5.1. 

With  our  off-site  ISP,  our  responsibility  included 
authoring  and  uploading  our  HTML  documents.  With 
an  on-site  host,  we  would  be  responsible  also  for  in¬ 
stalling  and  maintaining  the  service,  specifically  for 
managing  a  WWW  and  FTP  server. 

5.3.2  The  Skyway,  WWW,  HTML,  and  HTML 
Authoring 

Initially,  we  implemented  parts  of  the  Human  Fac¬ 
tors  Guide  on  the  World-Wide  Web.  WWW  provides 
adequate  support  for  the  text  and  graphics  in  this 
document.  Future  FARs,  reports,  etc.,  may  also  be 
published  in  WWW  format. 

Internet  users  work  with  Universal  Resources  Loca¬ 
tor  (URLs)  when  navigating  on  the  net.  URLs  func¬ 
tion  as  precise  addresses  by  which  Internet  resources 
are  located.  It  has  become  increasingly  common  for 
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organizations  to  include  a  central  WWW  URL  along 
with  their  standard  business  address.  The  current 
Skyway  URL  is: 


http://www.mindspring.com/-galaxy/skywayAtm! 

One  significant  advantage  of  the  WWW  is  its  wide¬ 
spread  availability.  Web  browsers  are  available  for 
most  common  platforms.  The  popular  MOSAIC 
viewer,  for  example,  is  available  for  MS-Windows, 
for  the  Macintosh,  and  for  UNIX  platforms. 

Authoring  the  HFG  WWW  version  (and  WWW  in¬ 
formation  in  general)  requires  utilizing  the  HTML 


format.  HTML  is  a  dialect  of  SGML;  a  much  larger 
specification.  HTML  is  a  simple  text-based  markup 
language  like  LaTeX  or  TROFF.  Much  HTML 
markup  work  is  done  manually.  While  this  method 
works  fine  for  typical  ‘pages,’  larger  document  data¬ 
bases,  such  as  the  Skyway,  require  a  more  sophisti¬ 
cated  and  scaleable  approach.  Since  Galaxy  primarily 
utilizes  Microsoft  Word  6.0  for  desktop  publishing, 
we  investigated  tools  that  directly  convert  Word  to 
HTML.  CU_HTML  is  one  such  tool;  it  meshes  well 
with  Word  6.0.  CU_HTML  uses  Word  6.0  templates 
and  macros  to  transform  Word  6.0  documents  auto¬ 
matically  into  the  HTML  format.  This  approach  is 
depicted  in  Figure  5.3. 
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CuiTcntly,  the  Skyway  consists  of  an  introductory  the  MOSAIC  page  introducing  the  Human  Factors 

Skyway  WWW  page  which  can  be  reached  from  any  Guide. 

Internet  Web  browser  using  the  URL: 


http://www«mindspring.con[i/-galaxy/sky  way.html 

Figure  5.4  shows  this  page  viewed  from  MOSAIC 
running  on  MS-Windows, 

There  are  two  hypertext  links.  One  link  takes  the  user 
to  the  Galaxy  Scientific  homepage;  the  other,  to  the 
Human  Factors  Guide.  Figure  5.5  (next  page)  shows 


Only  Chapter  1  is  present  now.  The  text  of  Chapter  1 
is  broken  into  several  subpages  for  general  hypertext 
organization  and  to  minimize  the  amount  of  time  a 
user  must  wait  while  information  is  being  down¬ 
loaded.  In  addition  to  the  text,  chapter  figures  and 
tables  can  be  found.  For  example,  Figure  5.6  (next 
page)  shows  the  MOSAIC  page  containing  one  par¬ 
ticular  graphic. 


47 


Figure's!  Gfaphic-fr6lTt:CliapS^^^  FactbcsM|; 


NCSA  ypsalc  forMS  Wiindows 
:  Annotate  Starting  Points  Personal 


DocwEaen^  Title: 


IIBL:  |http:/ywww.niindsprmg  com/^galaxy/f1  s1.html 


Figure  14.  Forward  fuselage  of  B-737  following  mfligjit  structural  failure  (NTSB). 
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Encoding  Chapter  1  of  the  Hiiinau  Factors  Guide  has 
shown  that  the  WWW  is  a  viable  medium  for  dis¬ 
seminating  information.  While  many  existing  WWW 
pages  are  quite  small,  our  effort  explores  issues  as¬ 
sociated  with  larger  documents.  Advantages  of 
WWW  publishing  include  world-wide  immediate  ac¬ 
cess,  multi-platform  support,  and  instantaneous  up¬ 
dates. 


We  surveyed  existing  public  aviation-  and  human 
factors-related  sources.  While  this  survey  is  in¬ 
complete,  if  for  no  other  reason  than  that  these 
sources  change  continuously,  the  results  provide  a 
glimpse  of  the  existing  electronic  landscape  and  indi¬ 
cate  the  existing  demand  in  this  area.  The  first  area 
we  explored  was  dial-up  Bulletin  Board  Systems 
(BBSs),  as  shown  in  Table  5.2.  We  then  surveyed 
existing  Aviation/Human  Factors  Internet-based 
services,  as  shown  in  Table  5.3  (next  page).  Finally, 
we  surveyed  Aviation/Human  Factors  CD-ROM  da¬ 
tabases,  as  shown  in  Table  5.4  (next  page). 


Name  of  Service 

Phone  # 

Airports  BBS 

(202)267-5205 

Air  Traffic  Operations  Service  BBS 

(202)267-5331 

(800)446-2777 

Air  Transport  Division  BBS 

(202)267-5231 

Pilot  Examiner  BBS 

(405)954-4530 

(800)954-4530 

FAA  Headquarters  BBS 

(202)267-5697 

Office  of  Environment  &  Energy  BBS 

(202)267-9647 

Navigation  and  Landing  BBS 

(202)267-6547 

Aviation  Rulemaking  Advisory  BBS 

(202)267-5948 

Orlando  FSDO  BBS 

(407)648-6963 

(407)648-6309 

(800)645-3736 

(800)645-FSDO 

Portland  MMEL  BBS 

(207)780-3297 

Safety  Data  Exchange  BBS 

(800)426-3814 

Aeromedical  Forum  BBS 

(202)366-7920 

Contel  Dual  User  Access  Terminal  System  (DUATS) 

(800)767-9989 

CompuServe  Information  Service  (CIS) 

GO  AVIATION  1 
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Type  of 
Service 

Reference 

Information  Provided 

WWW 

http://www.faa.gov 

FAA  Home  Page 

WWW 

http://www.dtic.dla.mil/iac/cseriac/iac.html 

CSERIAC  Home  Page 

WWW 

<unable  to  locate  at  this  time> 

Embry-Riddle  Avion  WWW  Site 

WWW 

http://www.virtual-airline.co.uk/virtual/  OR 
http://www.dennon.co.uk/virtual/ 

'The  Virtual  Airline" 

"Knowledge  and  Resources  for  the  Airline 
Industry." 

WWW 

http://www.sonic.net/aso/ 

Aircraft  Shopper  On-line 

WWW 

http://www.CdnAir.CA/ 

Airline:  Canadian  Air 

WWW 

http://www.iconz.co.nz/airnz/ 

airnz.htnni 

Airline:  Air  New  Zealand 

WWW 

http://www.winternet.com/~tela/ 

nwa-info.html 

Airline:  Northwest 

WWW 

http://www.winternet.com/~tela/ 

nwa.html 

Northwest  (travel  survey) 

WWW 

http://www/seanet.com/Bazar/ 
Aeroflot/Aeroflot. html 

Airline:  Aeroflot 

Usenet 

rec. aviation . 

(hierarchy  of  many  subgroups) 

Usenet 

sci. aeronautics 

Discussion  group  for  Aeronautics 

Usenet 

sci. aeronautics. airliners 

Airliner  Technology 

Mailing  lists 

listserv@cunyvm.cuny.edu 

Message  to  send:  subscribe  airline 

Discussion  Group  for  Airline  Industry 

CD-ROM 

Summit  Aviation 

Database  of  FARs,  ACs,  ADs 

CD-ROM 

ATP 

Database  of  FARS,JARs,  SBs 

CD-ROM 

ACS 

Database  of  FARS,JARs,  SBs 

CD-ROM 

Counterpoint  Publishing 

Database  of  CFR  and  FR 

We  need  more  analysis  to  determine  Skyway  re¬ 
quirements  accurately.  This  is  proceeding. 
Meanwhile,  the  WWW  is  proving  to  be  a  promising 
delivery  vehicle  for  digital  documentation.  Purely  as 
a  hypermedia  delivery  system,  it  works  well.  Ad¬ 
vanced  WWW  features  and  other  Internet  services 
promise  innovative  new  ways  to  integrate  and  engage 
the  Aviation  and  Human  Factors  community. 


Once  we  have  received  and  evaluated  more  User 
Needs  Surveys  and  obtained  a  clearer  picture  of 
our  user,  we  will  discuss  with  the  FAA  how  the  Sky¬ 
way  should  fit  into  the  overall  FAA  information  plan. 
Also,  we  are  in  the  process  of  implementing  the  next- 
generation  Skyway  node,  which  will  be  much  more 
powerful  and  flexible. 

Finally,  we  are  planning  the  next  set  of  Skyway 
services  including  archives,  newsletters  and  more  ex¬ 
perimental  services. 
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Chapter  5  -  Appendix 
Draft  of  User  Needs  Survey 


The  "Information  Skyway"  will  be  an  electronic  system  for  disseminating  safety-related  information  from  the 
Federal  Aviation  Administration  (FAA).  This  system  may  also  be  used  to  distribute  other  types  of  FAA- produced 
information,  such  as  regulations  concerning  commercial  and  general  aviation.  As  the  first  step  in  producing  this 
system,  Galaxy  Scientific  Corporation  is  conducting  a  survey  and  designing  a  proof-of-concept  prototype  for  the 
FAA.  The  survey  and  prototype  will  be  used  to  determine  the  feasibility  of  hosting  and  maintaining  an  on-ramp  to 
the  Information  Superhighway. 

Please  help  design  the  Information  Skyway  by  filling  out  this  survey.  The  data  from  this  survey  will  be  used  to 
determine  the  form  and  content  of  an  electronic  information  system  being  built  by  the  FAA  Office  of  Aviation 
Medicine.  The  information  obtained  from  this  survey  is  confidential,  and  you  do  not  need  to  identify  yourself. 

This  survey  is  designed  to  be  easy  to  fill  out  electronically;  for  multiple  choice  questions,  replace  the  character 
with  an  'X'.  For  questions  that  require  text,  just  type  your  answer  after  the  question. 

After  you  have  filled  out  this  survey,  please  return  it  to  Galaxy  Scientific.  E-Mail  is  preferred,  but  you  can  also 
return  it  via  fax  or  regular  mail. 

ATTN:  Electronic  Information  Survey 
Galaxy  Scientific  Corp. 

2310  Parklake  Drive  NE,  Suite  325 

Atlanta,  GA  30345 

phone:  404-491-1100 

fax:  404-491-0739 

email:  galaxy@mindspring.com 


— . .  Notice - 

This  information  collection  conforms  to  legal  and  administrative  standards  established  by  the  Federal  Government 
to  assure  confidential  treatment  of  statistical  information.  The  information  you  provide  will  be  used  only  for 
statistical  purposes  and  will  not  be  published  or  released  in  any  form  that  would  reveal  specific  information  reported 
by  an  individually  identifiable  respondent.  This  questionnaire  has  been  approved  by  the  Office  of  Management  and 
Budget,  and  has  been  given  0MB  Approval  Number  2120-0587. 

AGENCY  DISPLAY  OF  ESTIMATED  BURDEN: 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  five  minutes  per  response.  If 
you  wish  to  comment  on  the  accuracy  of  the  estimate  or  make  suggestions  for  reducing  this  burden,  please  direct 
your  comments  to  0MB  and  the  FAA  at  the  following  addresses: 

Office  of  Management  and  Budget  US  Department  of  Transportation 

Paperwork  Reduction  Project  Federal  Aviation  Administration 

MS  2120-0587  Office  of  Aviation  Medicine  A  AM-240 

Washington,  DC  20503  Washington,  DC  20503 
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Electronic  Information  Survey 


A.  INFORMATION  NEEDS 

1.  What  types  of  FAA-produced  aviation  information  do  you  currently  use?  (choose 
all  that  apply) 

_FARs 

_  Airworthiness  Directives 
_  Guidance  materials  (Advisory  Circulars,  etc.) 

_  Technical  publications 
_  General  Aviation  Airworthiness  Alerts 
_  Other  (please  describe  below) 

2.  What  FAA-produced  information  WOULD  you  use  if  given  easy  access? 

_  Regulations  (FARs,  Airworthiness  Directives,  etc.) 

_  Guidance  materials  (Advisory  Circulars,  etc.) 

_  Technical  publications 
_  General  Aviation  Airworthiness  Alerts 
_  Human  factors  information 
_  Other  (please  describe  below) 

3.  What  non-FAA  safety-related  aviation  information  do  you  currently  use? 

(choose  all  that  apply) 

_  Service  Bulletins 

_  Government  and  Commercial  Standards  (please  describe) 

_  Conference  proceedings  and  magazines 
_  Informal  discussions 
_  Other  (please  describe  below) 

4.  What  non-FAA  safety-related  aviation  information  WOULD  you  use  if  given  easy 
access?  (choose  all  that  apply) 

_  Service  Bulletins 

_  Government  and  Commercial  Standards  (please  describe) 

_  Conference  proceedings  and  magazines 
_  Informal  discussions 
_  Other  (please  describe  below) 

5.  What  computer  data  transfer  and  communications  hardware  do  you  have  access 
to? 

^CD-ROM 
_  Modem 
_  Internet 

6.  What  type  of  computer(s)  do  you  use? 

_  DOS  without  Windows 

_  DOS  with  Windows 
_  Macintosh 
^UNIX 
_  Mainframe 
_  Other  (please  describe) 

7.  What  aviation-related  electronic  resources  do  you  currently  use? 

_  FAA  bulletin  boards 
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_  Commercial  on-line  services  (America  On-line,  CompuServe,  etc.) 

__  CD  ROM-based  Commercial  Services  (Aircraft  Technical  Publications, 
Aviation  Compliance  Services,  Summit  Aviation,  etc.) 

_  Internet  newsgroups  and  mailing  lists 
_  Other  (please  describe  below) 

8.  Do  you  take  part  in  any  electronic  discussion  groups  related  to  aviation? 
_Yes 

_No 

9.  If  (8)  is  No,  would  you  take  part  in  any  aviation-related  electronic 
discussion  groups  if  you  had  access? 

_Yes 

_No 

10.  If  you  are  involved  in  General  Aviation,  what  electronic  information 
resources  would  you  use? 

_  Flight  training  material 
_  Maintenance  information 
_  Aviation  medicine 
_  Accident/incident  reports 
_  Other  (please  list  below) 

11.  Would  you  use  a  computer  to  submit  safety-related  information  if  you  had  a 
computer  and  appropriate  software? 

^Yes 

^No 

12.  Do  you  use  any  of  the  following  PC-based  flight  simulation  software? 

_  Microsoft  Flight  Simulator 

_  IFT-PRO 
_  AssureSoft 

_  FS-lOO  Desktop  Cockpit 
_  Other 


B.  OTHER  COMMENTS 

1 .  Describe  what  you  would  like  to  see  in  the  Information  Skyway. 

2.  What  do  you  like/dislike  about  existing  aviation-related  electronic  information  sources? 

3.  How  would  an  electronic  repository  of  safety-related  aviation  information  affect  your  decision  making? 
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C.  ABOUT  YOURSELF  (OPTIONAL) 

1.  Your  main  job  responsibility: 

_  Aviation  maintenance 

_  Researcher 
_  Student 
_  Pilot 

_  Document  management 
__  Regulatory 
_  Management 

_  Other  (please  describe  below) 

2.  Sector  of  your  work: 

_  Part  121  airline 

_  Part  135  airline 
_  General  aviation 
_  Military 

_  Government  (other  than  military) 

_  Academic 

_  Other  (please  describe  below) 

3.  What  is  your  most  advanced  pilot  certificate? 

_  Student 

_  Recreational 
_  Private 
_  Commercial 
_  Airline  Transport 
_  Certified  Flight  Instructor 
_None 

4.  Do  you  have  an  instrument  rating? 

_.Yes 

_No 

5.  About  how  many  TOTAL  flying  hours  do  you  have? 

6.  Contact  information  (may  be  used  to  gather  more  information,  but  will  not  be 
disclosed  or  distributed) 

Name: 

Address: 

City/State/Zip: 

Phone: 

Email: 


Thank  you  for  your  cooperation,  please  return  this  survey  to  Galaxy  Scientific 
Corp.  via  fax  or  E-Mail, 
fax:  404-491-0739 
email:  galaxy@mindspring.com 
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Colin  G.  Drury,  Ph.D.,  Caren  Levine  andJacqueline  L  Reynolds 
State  University  of  New  York  at  Buffalo 


This  project  was  initiated  to  provide  a  practical 
demonstration  of  human  factors/ergonomics 
implementation  in  an  airline  maintenance 
organization  and,  hence,  to  give  airlines  guidance  on 
implementing  their  own  programs.  Ergonomics,  and 
its  American  synonym  Human  Factors,  is  "the  science 
that  facilitates  maximum  human  productivity, 
consistent  quality,  and  long-term  worker  health  and 
safety"  (Burke,  1992).  Human  factors  measures  the 
job  demands  imposed  by  the  workplace,  environment, 
and  schedule.  It  then  compares  these  with  the 
workforce's  capabilities  to  meet  these  demands 
consistently.  Where  task  demands  exceed  human 
capabilities,  performance  will  break  down,  leading  to 
human  errors,  which  can  manifest  as  safety¬ 
compromising  incidents  and/or  on-the-job  injuries.  A 
better  (safer)  match  between  task  demands  and  human 
capabilities  can  be  achieved  by  changing  the  task 
demands  (workplace,  environment,  organization 
design),  by  changing  human  capabilities  (training, 
placement),  or  by  both.  Whether  the  organization's 
initial  motivation  for  the  human  factors  program  is 
public  safety,  improved  productivity,  or  reduced 
injuries,  the  analysis  is  the  same.  Indeed,  the  same 
analysis  can  be  used  to  specify  system  interventions, 
e.g.,  workplace  changes,  or  personnel  interventions, 
e.g.,  training. 

The  motivation  behind  the  current  project  arose 
specifically  from  human  factors  analyses  conducted 
in  1993  on  restrictive  spaces  in  aircraft  inspection 
tasks  (see  Reynolds  and  Drury,  1993).  As  part  of  that 
project,  on-the-job  injuries  (Oils)  analyzed  were 
found  to  be  space-related.  Hence,  when  we  sought  a 
site  for  demonstrating  human  factors/ergonomics 
intervention,  it  was  natural  to  choose  inspectors  and 
to  consider  OJI  reduction,  as  well  as  performance 
improvements,  i.e.,  error  reduction. 


The  human  factors  program  at  Northwest  Airlines 
was  created  with  the  mission  "to  redesign  work 
environments  to  prevent  on-the-job  injuries."  The 
program  was  initiated  by  the  formation  of  the  Human 
Factors  Task  Force  made  up  of  members  of  both 
management  and  the  hangar  workforce.  The  job  titles 
of  task  force  members  included  Safety  Manager,  lAM 
Safety  Representative,  Inspector,  Lead  Inspector,  and 
Northwest  Airlines  Process  Specialist  (Training 
Department).  Representatives  from  the  University  at 
Buffalo  were  assigned  to  act  as  task  force  advisors. 
The  initial  focus  of  the  program  was  the  inspection 
department  at  the  Atlanta  Maintenance  Base. 

The  inclusion  of  inspectors  on  this  task  force  was 
critical  to  its  potential  for  success.  Inspectors  have 
unparalleled  expertise  in  their  jobs  and  domain 
knowledge  that  leads  to  an  understanding  of  what 
changes  are  most  necessary  and  to  what  solutions  may 
or  may  not  work.  Inspectors  on  the  task  force  were 
encouraged  to  communicate  with  other  inspectors  and 
to  act  as  spokespeople  for  their  entire  crew. 

Typically,  inclusion  of  work  force  representatives  in 
analysis  and  redesign  of  their  own  jobs  makes  them 
more  inclined  to  accept  ergonomic  solutions  task 
force  implements.  This  is  because  they  actively 
contributed  to  the  solution-development  process. 

6.1.1  Task  Force  Objectives  and  Guidelines 

Burke  (1992)  emphasizes  the  benefits  that  can  be 
obtained  when  a  human  factors  task  force  addresses 
human  factors  issues  within  an  organization.  A  team 
approach  gives  the  organization  maximum  input  from 
various  people  who  will  be  affected  by  any  changes. 
For  a  task  group  to  be  successful,  its  members  must 
be  comfortable  working  together  and  must  fully 
understand  the  importance  of  their  commitment  and 
contribution  to  the  task  force.  In  recent  years. 
Northwest  Airlines  has  emphasized  team  activities. 
There  are  well-established  procedures  for  teams  to 
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form,  gain  confidence,  organize  their  activities,  and 
implement  their  findings. 

The  initial  objectives  of  the  Northwest  Airlines 
Human  Factors  Task  Force  were  as  follows: 

1 .  Develop  a  process  for  identifying  and  addressing 
ergonomics  issues  within  the  inspection 
department  that  could  later  be  expanded  to  all 
Northwest  Airlines  departments 

2.  Involve  employees  in  the  ergonomics  process 

3 .  Reduce  the  number  of  Oils 

4.  Develop  ergonomic  solutions  that  could  be 
implemented,  with  results  that  could  be  measured 

5.  Teach  employees  about  ergonomics,  so  they 
could  help  widen  the  task  force’s  focus 

6.  Commit  to  transfer  the  technology  and  the 
processes  this  task  force  used  to  other  areas  at 
Northwest  Airlines. 

The  task  force’s  guidelines  were  as  follows: 

1 .  Focus  on  inspection  jobs  and  tasks  in  the  hangar 
area 

2.  Identify  the  jobs  and  tasks  to  analyze 

3.  Establish  an  action  plan  to  effect  short-  and  long¬ 
term  improvements 

4.  Members  should  commit  to  a  one-year 
participation  in  the  task  force 

5.  The  group  leader  to  be  elected  by  the  entire  task 
force 

6.  A  task  force  member  may  work  on  this  project  up 
to  100%  of  his  or  her  time 

7.  After  its  initial  start-up  meetings,  the  task  force 
will  establish  its  own  agenda 

8.  The  group  leader  will  communicate  a  weekly 
report  to  all  task  force  members. 

6.1.2  Program  Development 

The  steps  that  the  Northwest  Airlines  Human  Factors 
Task  Force  took  closely  followed  the  seven  general 
steps  in  an  ergonomic  process,  as  described  by  Burke 
(1992). 

1 .  Determine  the  measurement  criteria  and  target 
the  jobs  to  be  studied. 

©  Determine  which  areas  should  be 
targeted  for  analysis  and  intervention. 

®  Choose  the  specific  criteria  which  will 
help  determine  target  areas,  e.g.,  injury 
rate. 

2.  Gather  job  background  information. 

o  Document  the  job  to  be  analyzed, 

including  the  job  description,  the  tools 


necessary  to  perform  the  job,  physical 
dimensions  of  the  workspace,  etc. 

3.  Identify  ergonomic  risk  factors. 

©  Identify  conditions  likely  to  act  as 
barriers  to  optimal  productivity  and 
consistent  quality  and/or  that  have  been 
associated  with  a  high  incidence  of 
injuries. 

4.  Discover  ergonomic  interventions. 

©  Brainstorm  about  all  possible 
interventions  to  each  risk  factor, 
considering  the  following: 

-  changing  inputs/materials 

-  changing  output/product 

-  changing 

machine/environment 

-  changing  procedures  dealing 

with  workers,  e.g., 
training. 

5.  Screen  interventions. 

o  Choose  interventions  to  implement 
based  on  decision  criteria  such  as  cost, 
benefits,  utility,  consequences  of  no 
action,  injury  rate,  etc. 

6.  Implement  interventions. 

®  Orient  those  affected  about  why  the 
intervention  was  chosen,  what  its 
expected  impact  is,  and  who  to  contact 
with  questions/comments/concerns. 

7.  Track  the  effectiveness  of  the  interventions. 

®  Assess  each  intervention's  effectiveness 
and  decide  whether  to  expand,  amend, 
alter,  or  abandon  the  particular 
intervention. 

Once  the  human  factors  task  force  was  selected,  it 
was  necessary  to  educate  its  members  about  what 
human  factors  is  and  how  human  factors  can  be  used 
to  improve  the  workplace.  The  University  at  Buffalo 
conducted  a  one-day  training  seminar,  using  materials 
developed  from  previous  FAA/AAM  projects  and 
ICAO’s  SHEL  model  of  human  factors.  The  training 
specifically  built  on  the  University  at  Buffalo’s 
previous  involvement  with  Northwest  Airlines’ 
Atlanta  Maintenance  Base  and  its  inspection 
activities. 

The  task  force  selected  jobs  to  be  analyzed  in  the  first 
phase  of  the  human  factors  program.  The  following 
jobs  were  identified  by  inspectors  as  five  of  the 
longest,  most-difficult  inspection  tasks:  Electrical  and 
Equipment  Compartment  Inspection  (E&E 
Compartment),  Keel  Inspection,  Fuel  Tank 
Inspection,  Combustion  Chamber  Inspection  (PS4 
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Drain  Box),  and  Nose  and  Forward  Accessory  From  this  list,  task  force  members  took  responsibility 

Compartment  Inspection  (Forward  Access  for  pursuing  specific  potential  solutions  in  the 

Compartment).  following  areas: 


Four  of  the  jobs  were  analyzed  using  the  electronic 
inspection  audit  program  the  University  at  Buffalo 
developed  (see  Koli  and  Drury,  1995).  Inspectors  on 
the  task  force  conducted  the  audits.  The  audit  results 
for  the  keel  inspection  are  provided  in  Appendix  6-A 
as  an  example  of  their  work.  To  progress  from 
analysis  to  redesign  for  each  of  these  audits,  a  list  of 
ergonomic  risk  factors  was  identified.  A  few  risk 
factors  from  all  four  tasks  were  combined  into 
problems  with  workcards  and  problems  with  lighting. 
The  list  of  ergonomic  risk  factors  for  each  area  is 
included  in  Appendix  6-B.  The  nominal  group 
technique  was  utilized  to  rank  each  risk  factor  for 
each  of  the  six  main  areas,  for  four  specific  tasks 
(E&E  compartment,  forward  accessory  compartment, 
keel,  and  PS4  Drain  Box),  and  for  two  general  areas 
(workcards  and  lighting). 

As  follows,  the  three  risk  factors  with  the  highest 
rankings  were  chosen  for  closer  study  in  each  of  the 
six  areas: 

•  Workcards 

•  Card  content  inaccurate 

•  Breaks  between  cards  inappropriate 

•  Card  contrast  varied 

•  Lighting 

•  Fixtures  dirty 

•  Lighting  inadequate  at  the  back  of  the 
hangar 

®  No  preventive  maintenance  program  for 
lighting 

•  Keel  Inspection 

•  Body  positioning 

•  Cleaning 

•  Lighting 

•  PS4  Drain  Box  Inspection 

•  Body  positioning 

®  NDT  equipment 

•  Cleaning 

o  E&E  Compartment  Inspection 
®  Lighting 

®  Temperature 

®  Equipment 

»  Forward  Access  Compartment  Inspection 
®  Ladder  design 

•  Ladder  control 

»  Work  planning 


®  Improved  cleaning 
®  Ladder  purchase  and  control 
®  Workcard  design 

®  Improved  task  lighting. 

An  action  plan  provided  a  time  line  for  these 
activities  that  ensured  analysis,  implementation,  and 
measurement  of  results  within  the  time  frame  of  this 
FAA/AAM  project. 

6.1.3  Redirection  of  the  Ergonomics  Task  Force 

Initially,  the  Task  Force  followed-up  on  members’ 
assignments  to  track  progress  according  to  the  action 
plan.  However,  it  became  apparent  that  the  Task 
Force  as  a  whole  was  not  progressing  on  developing 
solutions,  as  agreed.  The  researchers  met  with  the 
task  force  and  management  to  learn  the  reasons  for 
the  lack  of  progress  and  to  help  develop  alternative 
strategies. 

A  number  of  factors  that  had  not  prevented  progress 
in  team  formation,  job  analysis,  and  solution 
generation  surfaced  when  it  was  time  for 
implementation. 

1 .  The  workforce  members  of  the  task  force  felt  that 
they  had  no  mandate  to  pursue  their  assignments 
as  part  of  their  busy  schedules. 

2.  Some  of  the  solutions  had,  or  appeared  to  have 
had,  implications  beyond  the  Task  Force’s 
control.  For  example,  workcard  design  is  a 
headquarters  function,  not  easily  controlled  or 
changed  at  a  remote  base. 

3.  Other  solutions  required  expenditure,  e.g.,  task 
lighting,  which  was  not  immediately  seen  as 
available  in  the  current  fiscal  climate. 

4.  Perhaps  most  importantly,  although  task  force 
members  were  opinion  leaders  within  their 
groups,  and  a  senior  management  person  acted  as 
“champion”  of  the  effort,  neither  management 
nor  the  workforce  felt  a  groundswell  of  support 
for  the  Task  Force’s  activities. 

For  these  reasons,  the  task  force  was  disbanded,  and 
the  ergonomics  efforts  were  refocused  on  a  different 
problem  that  could  have  broad-based  support  and  be 
entirely  under  control  at  the  maintenance  base. 
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Specifically,  many  task  force  members  recognized 
communication  between  shifts  as  one  area  in  need  of 
improvement.  Also,  communication  between  shifts 
needed  no  ‘outside’  assistance  to  implement  a 
solution.  Instead  of  having  task  force  members 
implement  the  ergonomics  audit  program,  which 
worked  very  well  to  identify  human  factors  problems, 
a  broad-based  instrument  was  designed  to  obtain 
input  about  communication  issues  from  inspectors  on 
all  three  shifts.  We  reasoned  that  such  input  would 
produce  buy-in  to  potential  solutions,  thus  easing 
implementation.  Of  course,  broader  participation 
meant  that  expectations  would  be  raised  for  more 
people,  forcing  at  least  some  implementation  if 
management/workforce  trust  was  to  be  preserved. 
Fortunately,  improving  communication  of  technical 
information  between  participants  has  a  good  history 
in  human  factors  generally,  and  in  aircraft 
maintenance  specifically  (see  Taylor,  1992). 


As  in  any  industry,  effective  communication 
within  an  organizational  unit,  and  among 
organizational  units,  is  critical  for  maintaining 
productivity  in  airline  inspection  and  maintenance. 
Taylor  (1992)  writes,  “Effective  communication  is  no 
longer  limited  to  merely  acquiring  the  information 
that  an  individual  needs  to  make  decisions. 
Communication  is  increasingly  a  systems  issue — ^it  is 
inextricably  bound  to  cooperation,  coordination,  and 
otherwise  working  together  in  a  joint  task  or  job  for 
which  individuals  cannot  succeed  by  working 
separately.”  Airline  inspectors  often  help  drive  the 
heavy  maintenance  of  aircraft.  They  are  the  first  to 
look  over  an  aircraft  and  have  the  task  of  identifying 
all  the  problems  with  it.  Inspectors  decide  which 
problems  maintenance  must  fix  before  an  aircraft  can 
leave  the  hangar,  as  well  as  which  problems  can  be 
delayed  until  the  next  maintenance  check.  After  the 
maintenance  work  is  performed,  inspectors  must 
ensure  that  it  was  done  properly.  An  aircraft  cannot 
leave  the  hangar  until  all  work  is  signed  off  by  the 
appropriate  authority,  usually  the  inspectors.  An 
inspector  must  be  able  to  share  information  with 
management  and  other  employees  so  that  everyone 
understands  an  aircraft’s  current  status.  At  Northwest 
Airlines’  Atlanta  Maintenance  Base,  for  example,  an 
inspector  may  find  it  necessary  to  communicate  with 
the  following  people: 

o  other  inspectors  on  the  same  shift 
o  inspectors  on  the  two  other  shifts 


o  mechanics 
®  the  lead  inspector 

9  the  inspection  manager 

®  the  maintenance  manager 

o  engineers 
»  other  management 
9  the  flight  crew. 

The  inspector  must  have  the  communication  tools  and 
skills  to  share  information  with  other  members  of  the 
organization,  as  necessary.  Although  communication 
is  an  important  aspect  of  aircraft  maintenance,  it  fails 
at  times.  To  understand  possible  failure  modes,  a 
national  source  of  error  data  (Aviation  Safety 
Reporting  System,  or  ASRS)  was  analyzed 
specifically  to  identify  communication  errors  in  the 
maintenance  environment. 

6.2.1  Typical  Airline  Industry  Commnnication 
Problems  (ASRS  Reports) 

Fortunately,  human  errors  in  aircraft  maintenance  are 
rare.  Since  errors  are  unlikely  to  be  observed  during  a 
study  such  as  ours,  possible  errors  must  be  inferred 
from  other  sources.  A  review  of  NASA's  ASRS 
mechanic  reports  identified  that  serious  consequences 
can  occur  when  inspectors  and  mechanics  are  unable 
to  communicate  efficiently  with  their  co-workers.  It  is 
important  to  remember  that  ASRS  reports  are 
reported  by  individuals  on  a  voluntary  basis.  In  many 
cases,  the  reports  have  not  been  corroborated  by  the 
FAA  or  NTSB,  and  the  data  cannot  be  used  to  infer 
the  prevalence  of  a  particular  problem  within  the 
national  aviation  system.  The  incidents  discussed  here 
occurred  over  many  years  (January,  1987-February, 
1994)  at  many  airlines.  They  are  not  Northwest 
Airlines  incidents. 

Some  common  communication  problems  present 
themselves  upon  a  close  review  of  the  ASRS  reports. 
First,  many  incidents  are  caused  by  mechanics 
becoming  distracted  in  the  middle  of  performing  a 
task.  Mechanics  often  do  not  write  down  what  they 
have  accomplished,  or  what  parts  of  a  task  need  to  be 
completed.  At  times,  a  mechanic  may  have  to  allow 
someone  else  to  finish  a  task.  This  may  lead  to 
difficulties  when  the  second  mechanic  does  not 
clearly  understand  the  situation  or  does  not  realize 
specifically  what  remains  to  be  done.  Other  times,  a 
mechanic  intends  to  come  back  and  finish  a  task  but 
forgets  that  the  task  was  not  completed.  This  could 
lead  to  serious  problems  if  the  uncompleted  task  is 
not  detected  before  the  plane  takes  off.  This  type  of 
problem  may  also  occur  at  shift  changes,  when 
mechanics  cannot  finish  a  task  before  their  shift  ends. 
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The  next  shift  assigned  to  finish  the  work  may  not 
clearly  understand  where  the  previous  shift  left  off. 

This  may  result  in  duplication  of  effort  on  some  tasks 
or,  more  seriously,  the  omission  of  some  tasks 
completely,  e.g.,  the  second  shift  assumes  that  the 
previous  shift  has  performed  a  certain  task  and  does 
not  verify  this  to  be  the  case. 

I  was  assigned  to  aircraft  work 
release  items.. ..I  was  in  the  process 
of  reinstalling  the  plug  and  covers 
for  the  turbine  section  when  another 
mechanic  asked  if  I  needed  any  help. 

I  asked  him  if  he  would  install  the 
ignitors.  I  saw  him  install  the 
outboard  ignitor.  Then  he  went 
under  the  engine  to  what  I  thought 
was  to  install  the  inboard  ignitor. 

While  he  was  under  the  engine,  I 
saw  him  install  the  screen  back  on 
the  starter,  but  I  did  not  go  back  and 
check  his  work,  because  I  trust  the 
work  he  does.  I  am  the  one  who 
signed  off  the  block  on  the 
paperwork.... The  inboard  ignitor  was 
never  installed.  (ACN  #250135) 

Another  mechanic  was  assigned  the 
open  and  close  of  the  engine.  He 
opened  all  plug  panels  and  ignitors.  I 
stopped  to  help  him  close  the  engine. 

I  installed  the  outboard  ignitor  and 
installed  the  starter  air  deflector, 
only  per  maintenance  manual  72. 

The  inboard  ignitor  was  never 
installed.  I  did  not  know  [if]  the 
inboard  ignitor  was  left  out,  or  [was 
even]  out  at  all.  (ACN  #250330) 

Another  problem,  somewhat  related  to  the  problem 
described  above,  is  that  generally  one  mechanic  must 
sign  off  on  the  completion  of  a  task,  although  more 
than  one  person  may  have  actually  worked  on  the 
task.  Thus,  it  is  difficult  to  pinpoint  who  actually 
completed  the  work  when  a  problem  arises.  The 
mechanics  who  assisted  may  later  forget,  or  deny,  that 
they  participated  in  completing  the  task  in  question. 

Oil  was  serviced  to  full  by  another 
mechanic.  However,  he  was 
reassigned  to  another  aircraft  before 
completing  the  log  entry.  At 
departure  time,  I  completed  [the] 
maintenance  sign  off  in  [the] 
logbook.  The  oil  tank  cap  apparently 


was  not  latched  in  the  closed 
position.  (ACN  #245568) 

Maintenance  inspected  [the]  aircraft 
and  found  all  six  securing  screws 
missing  from  left-hand  most 
outboard  wing  trailing  edge 
panel.... Further  investigation  shows 
several  individuals  were  involved  in 
the  close-up  of  the  aircraft  at 
completion  of  the  check,  but  no  one 
person  assumes  responsibility  or  full 
knowledge  of  this  one  particular 
panel.  However,  there  is  a  signature 
of  a  supervisor  who  specifically 
signed  stating,  “All  panels  were 
secured.”  (ACN  #101899) 

A  third  problem  occurs  when  mechanics  are  given 
incorrect  verbal  descriptions  of  discrepancies  or 
descriptions  varying  from  the  written  description  in 
the  log  book.  Similarly,  a  mechanic  can  be  assigned 
to  perform  a  task  without  receiving  all  the  correct 
paperwork  which  accompanies  the  task.  This  can  lead 
to  the  mechanic  making  an  incorrect  diagnosis  of  the 
problem  and,  consequently,  taking  incorrect  action  to 
correct  the  problem.  In  some  cases,  inaccurate 
diagnosis  led  a  mechanic  incorrectly  to  defer 
maintenance  that  should  have  been  completed 
immediately. 

I  was  told  [verbally]  that  the  roll 
spoiler  outboard  ground  caution 
light  was  illuminated.  I  sent  an  A&P 
down  to  check  [it]  out  and  defer  the 
system.  He  was  unable  to  duplicate 
any  problem,  but  we,  by  phone 
conversation,  decided  to  defer  the 
system  in  case  the  pilots  had  a 
problem  on  the  morning  departure. 

[Later,]  when  reviewing  the 
logbook,  I  discovered  I  had  been 
given  wrong  information  from 
maintenance  control  about  which 
light  had  illuminated.  The  roll 
spoiler  outboard  hydraulic  light  was 
the  light  that  actually  was  written  up, 
and  this  would  not  be  something  you 
would  defer.  (ACN  #243444) 

These  problems  emphasize  the  importance  of  written 
communication  in  the  airline  industry.  Verbal 
communication,  although  often  more  convenient,  is 
more  error-prone,  especially  when  information  must 
be  remembered  for  long  periods  of  time  or  must  be 
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passed  sequentially  through  a  number  of  people.  The 
"telephone”  game  provides  a  good  example  of  this 
problem:  as  information  is  passed  from  one  person  to 
another,  the  message  tends  to  become  increasingly 
confused.  Written  communication  can  serve  as  a 
permanent  record  of  events  and  is  less  subject  to  the 
frailties  of  human  memory.  However,  since  written 
records  may  be  used  as  an  investigative  tool  to  prove 
the  actions  a  maintenance  crew  took,  workers  may 
feel,  “It  gives  them  something  to  hang  you  with!” 
There  is  understandable  reluctance  in  all  branches  of 
the  airline  industry  to  write  anything  not  specifically 
required  to  be  committed  to  paper. 

6.2. 1.1  Summary  of  Communication  Failures 

Table  6. 1  presents  the  types  of  communication 
failures  contributing  to  the  incidents  reported  in  the 
ASRS  database.  The  data  in  this  table  are 
representative  only  of  the  twenty-eight  ASRS  reports 
we  analyzed. 

Type  of  Failure: 

®  F  =  failure  to  communicate 
&  V  =  verbal  communication  wrong/inadequate 
e  W  =  written  communication  wrong/inadequate 
®  M  =  memory  failure  (forgot  to  do  something) 
(See  Table  6.7) 

6.2.2  Identification  of  Communication  Problems 
Within  The  Inspection  Department 

Table  6.1  shows  that  certain  failure  types  are 
associated  with  different  communication  needs. 
While  ASRS  data  is  not  a  statistically  valid  random 
sample  of  errors,  it  can  be  used  to  identify  forms  of 
failure. 

Obviously,  a  mechanic  communicating  with  himself 
or  herself  at  a  later  time  can  have  a  memory  failure 
(M).  When  this  happens,  the  mechanic  usually  relied 
on  memory  rather  than  a  written  note  or  a  job  aid, 


such  as  a  checklist,  that  would  have  prevented 
memory  failure.  Mechanics  communicating  with 
flight  crew  are  subject  to  failures  of  both  written  (W) 
and  verbal  communication  (V).  Communication 
problems  in  the  opposite  direction,  i.e.,  from  flight 
crew  to  maintenance,  are  either  failures  to 
communicate  at  all  (F)  or  a  breakdown  of  the  written 
process  (W).  Perhaps  this  results  from  the  widely 
different  background  training  of  Flight  Operations 
and  Technical  Operations  and  the  lack  of 
opportunities  for  verbal  communication  between 
these  groups.  Clearly,  methods  of  improving 
communications  between  these  groups  are  needed, 
e.g.,  extensions  of  CRM  and  MRM  to  joint  training. 

Communication  problems  between  mechanics,  and 
between  mechanics  and  supervisors,  are  all  either 
failure  to  communicate  at  all  (F)  or  a  failure  of  verbal 
communication  (V).  This  also  includes  shift  change 
communication  in  the  final  column  of  Table  6.1. 
Clearly,  written  communication  does  not  fail;  if 
people  use  written  communication,  then  this  is 
adequate.  The  main  emphasis  for  addressing  these 
problems  should  be  ensuring  that  mechanics  and 
supervisors  use  written  communication.  Thus,  the 
new  focus  of  this  project  became  redesigning 
communication  forms  so  mechanics  and  supervisors 
can  use  them  more  easily. 

Since  communication  is  critical  to  the  successful 
performance  of  airline  inspectors,  we  decided  to 
examine  the  communication  system  for  inspectors 
currently  in  place  at  the  Atlanta  Maintenance  Base  to 
see  if  improvements  could  be  made.  We  expected  that 
an  inspector’s  (or  a  mechanic’s,  or  a  supervisor’s) 
effectiveness  can  be  improved  by  providing  better 
communication  tools  that  make  it  easier  to  collect 
necessary  information  and  to  pass  that  information  to 
other  supervisors  and  mechanics. 

After  interviewing  many  inspectors,  it  was  obvious 
that  each  inspector  views  the  job  (and  the  larger 
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system)  differently.  The  shift  on  which  the  inspector 
usually  works  (and  thus  the  inspector's  lead 
inspector),  as  well  as  years  experience  as  an 
inspector,  are  just  two  factors  that  appear  to  affect 
each  inspector's  perceptions.  Due  to  such  wide 
variations  among  inspectors,  we  decided  to  question 
all  inspectors  to  gain  a  broad  view  of  the  actual 
communication  system  in  the  inspection  department. 
The  user  needs  analysis  was  designed  to  identify  tools 
currently  supporting  communication  within  the 
inspection  department  and  between  inspectors  and 
other  departments.  The  user  needs  analysis  we  used  is 
included  as  Appendix  6-C.  As  a  follow-up  to  the 
communication  user  needs  analysis,  we  conducted 
further  personal  interviews  with  many  inspectors. 
These  interviews  did  not  follow  any  pre-defined 
format;  their  purpose  was  simply  to  allow  inspectors 
to  talk  about  communication  issues  at  Northwest 
Airlines  and  to  provide  background  information  to 
help  interpret  the  user  needs  analysis  responses. 

A  particular  focus  of  the  communication  user  needs 
analysis  was  the  shift  turnover  log.  Currently,  the  shift 
turnover  log  is  a  bound  book  with  numbered  pages. 
Entries  are  made  in  the  log  each  day,  usually  by  the 
lead  inspectors.  Information  in  the  log  includes 
personnel  issues,  e.g.,  who  called  in  sick,  who  left 
early,  who  is  working  overtime,  etc.,  and  aircraft 
issues,  which  are  usually  only  a  quick  summary  of 
each  aircraft’s  status,  e.g.,  in  buy-back,  shakedown, 
etc.  An  entry  occasionally  includes  a  description  of  a 
problem  an  inspector  encountered  during  the  shift.  It 
is  difficult  to  identify  who  made  an  entry  in  the  log, 
and  few  entries  are  ever  followed-up  with  another 
entry  describing  how  the  problem  was  resolved.  The 
existing  shift  turnover  log  does  not  serve  as  a 
communication  tool,  showing  the  tasks  with  multi¬ 
shift  implications,  nor  does  it  provide  the  information 
necessary  for  subsequent  shifts  to  "pick  up"  where  a 
previous  shift  left  off.  Thus,  our  communication  user 
needs  analysis  was  designed  to  identify  whether 
inspectors  use  the  existing  shift  turnover  log  as  a 
helpful  source  of  information  and/or  whether  a 
different  type  of  log  would  better  serve  inspectors' 
needs. 

6.2.3  Results  from  the  Communication  User  Needs 
Analysis 

We  received  17  responses  to  our  user  needs  analysis 
from  the  approximately  30  inspectors  at  the 
Maintenance  Base.  User  needs  analysis  responses  are 
summarized  in  Table  6.2  (next  page). 


User  needs  analysis  responses  identified  a  general 
problem  with  inspectors' job  satisfaction.  Many 
inspectors  report  having  difficulty  obtaining 
information  they  need  to  perform  the  job.  They  are 
unwilling  to  share  information  with  others,  unless  it  is 
absolutely  necessary.  This  reluctance  to  communicate 
is  a  serious  problem  and  must  be  addressed  if 
inspection  productivity  is  to  be  improved.  The 
inspectors  also  identified  shortcomings  in  the 
communication  system  at  Northwest.  Inspectors  do 
not  use  the  shift  turnover  log  regularly,  almost  always 
need  to  search  for  more  information  after  being 
assigned  a  job,  have  experienced  on-the-job  problems 
caused  by  miscommunication,  and  deal  with  each 
other  almost  always  verbally.  The  shift  turnover  log  is 
seen  as  a  managerial  tool,  not  as  a  way  to 
communicate. 

It  is  important  to  note  that  the  average  years  of 
experience  of  inspectors  responding  to  the  user  needs 
analysis  is  6.6,  with  a  standard  deviation  of  3.6. 
Previous  studies  have  indicated  that  it  is  common  in 
the  aircraft  industry  to  have  mechanics  with  long 
service  and  with  very  short  service,  with  very  few  in 
the  middle  (Taylor,  1990).  At  the  Atlanta 
Maintenance  Base,  the  less-experienced  inspectors 
tended  to  return  completed  user  needs  analysis  (over 
half  had  only  3-5  years  experience);  our  results  reflect 
their  particular  dissatisfaction  with  the  current 
communication  system.  This  result  is  not  altogether 
unexpected.  Experience  as  an  inspector  often  means 
increased  knowledge,  information,  and  familiarity. 
Less-experienced  inspectors  may  require  more 
external  information  to  perform  a  task  (they  cannot  so 
easily  rely  on  internal  knowledge)  than  more 
experienced  inspectors.  Less-experienced  inspectors 
also  may  be  less  able  to  respond  to  verbal  instructions 
and  information.  Therefore,  they  may  be  less  satisfied 
with,  and  more  able  to  recognize  problems  in,  current 
modes  of  communication.  Experienced  inspectors  are 
accustomed  to  the  way  things  are  done  and  may  be 
reluctant  change.  Our  results  may  reflect  a 
communication  system  designed  to  meet  the  needs  of 
experienced  inspectors,  and  of  those  with  managerial 
responsibilities,  while  de-emphasizing  the  increased 
information  demands  of  those  with  less  experience. 

User  needs  analysis  responses  also  indicate  that  many 
inspectors  perceive  a  lack  of  what  is  termed 
situational  awareness  in  human  factors;  they  do  not 
understand  how  their  specific  tasks  fit  into  the  larger 
picture  of  airline  maintenance.  Inspectors  may  be 
unaware  of  what  is  happening  beyond  their  own  work 
assignments  and  of  how  their  assignments  affect  (and 
are  affected  by)  other  departments.  For  example,  jobs 
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Number  of  Years  Experience 
Sources  of  Information 


Nature  of  Information  Received 


Destinations  of  Information 


Nature  of  Passed  Information 
Is  All  Information  Available 


Do  You  Read  Shift  Turnover  Log?  How  Often? 


Do  You  Write  in  Shift  Turnover  Log?  How  Often? 


Purpose  of  Shift  Turnover  Log? 


Information  to  be  Included  in  Shift  Turnover  Log? 


Should  STL  Be  On  Sceptre,  a  Book,  or  Both? 


Time  Between  Shifts,  and  Is  It  Sufficient? 


Attendance  at  Regular  Crew  Meetings? 


Are  Regular  Crew  Meetings  Beneficial? 


Problem  Caused  by  Miscommunication? 


_ of  Rosylte _ 

average  =  6.6  years 

median  =  5  years _ 

lead  inspector,  manuals,  managers  (inspection, 
maintenance),  engineering,  other  inspectors _ 


mostly  verbal,  some  written _ 

lead  inspector,  mechanics,  other  inspectors, 
managers  (inspection,  maintenance),  anyone  who 

asks _ 

mostly  verbal,  some  written _ 

No  - 1 1  (65%) 

Yes -5  (29%) 

NA-1  (6%) 


No  -  9  (53%) 

Yes,  When  Acting  Lead  -  5  (29%) 

Yes,  (Almost)  Every  Day  -  3  (18%) _ 

No  -  9  (53%) 

Yes,  When  Acting  Lead  -  3  (1 8%) 

Yes,  When  A  Problem  Arises  -  2  (11%) 

Yes,  (Almost)  Every  Day  -  3  (18%) 


lead  turnover  information,  personnel  notes,  status 
of  aircraft,  communication  between  shifts,  written 

account  of  daily  activities _ 

more  information  about  the  aircraft 
status  of  long-term  projects 
more  cautions  and  warnings 
important  work  in  progress 

what  tasks  managers  want  done  on  a  shift _ 

Book  Only  - 1 1  (65%) 

Sceptre  -  2  (12%) 

NA  -  4  (23%) 


Time  Is  Sufficient  [5-20  min.]  - 12  (70.5%) 

No  Time  Needed  -  4  (23.5%) 

Time  Is  Sufficient,  But  Inefficiently  Used  - 1  (6%) 
No -7  (41%) 

Yes  -  8  (47%) 

NA-2  (12%) _ 

No  -  4  (23.5%) 

Yes  -  8  (47%) 

Sometimes  -  4  (23.5%) 

NA  - 1  (6%) _ 

No -3  (18%) 

Yes  - 10  (59%) 

NA  -  4  (23%) 


are  often  assigned  to  inspectors  in  what  they  perceive 
as  a  random  manner,  e.g.,  large  jobs  may  be  assigned 
only  early  in  a  shift,  more  difficult  jobs  may  be 
delayed  until  easier  ones  are  completed,  etc.  Many 
times,  there  seems  to  be  little  consideration  of  how 
job  scheduling  affects  the  maintenance  department. 


6.2.4  Results  From  Personal  Interviews  with 
Inspectors 

During  site  visits  to  the  Atlanta  Maintenance  Base, 
we  spoke  personally  with  many  inspectors  about 
communication  at  Northwest  Airlines.  These 
conversations  generally  support  the  results  from  the 
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user  needs  analysis,  although  they  provide  more 
insight  into  inspectors'  specific  communication  needs. 
Some  points  inspectors  made  in  these  conversations 
include  the  following. 

1 .  Inspectors  acknowledge  that  they  almost  always 
communicate  verbally  with  their  lead  inspector  and 
with  other  Northwest  employees.  Most  inspectors  had 
never  really  considered  the  consequences  if,  at  some 
later  time,  there  was  a  problem  with  an  inspection 
they  conducted.  Although  workcards  and  non-routine 
cards  provide  a  written  account  of  the  completed 
tasks,  there  is  important,  not  legally  required 
information  that  is  never  permanently  recorded. 
Without  written  records,  it  is  impossible  to  remember 
exactly  what  occurred  and  what  steps  had  been  taken. 
Even  if  an  inspector  did  everything  correctly,  there 
would  be  no  way  to  prove  this  in  an  investigation. 

The  following  incident  is  taken  from  the  ASRS 
database:  this  is  not  data  collected  at  Northwest 
Airlines.  It  illustrates  the  potential  danger  in  failing  to 
maintain  accurate  written  records  of  all  maintenance 
activities. 

A  ‘visiting’  mechanic  was  assigned 
to  repair  an  engine.  While 
performing  the  work,  he  accidentally 
dropped  a  rag  into  the  gearbox 
cavity.  After  searching, 

unsuccessfully,  for  the  rag,  the 
mechanic  notified  (verbally)  the  lead 
mechanic  of  the  problem.  The  lead 
mechanic  ordered  a  horoscope  of  the 
engine,  which  did  not  show  that  the 
rag  was  inside.  Although  the 
mechanic  continued  to  say  that  the 
rag  was  still  inside  the  engine,  the 
lead  mechanic  ordered  that  the  repair 
be  completed  so  that  the  plane  could 
be  released  for  a  flight.  The 
mechanic  was  sent  home  before  the 
leak  check  on  the  engine  was 
completed.  On  its  initial  flight,  the 
plane  was  forced  to  turn  back  to  the 
originating  airport  due  to  a  low  oil 
pressure  warning.  The  engine  was 
removed  for  further  repair.  During 
the  investigation,  the  rag  was  found 
to  have  clogged  the  scavenge  pump 
filter  screen.  The  mechanic  was 
interviewed  twice  by  airline  quality 
assurance,  and  the  incident  was 
written  up  in  a  report  submitted  to 
theFAA.  (ACN  #233249) 


From  an  analysis  of  this  incident  it  is  clear  that:  if  the 
mechanic  had  made  a  written  entry  in  the  maintenance 
log  concerning  this  incident,  there  would  have  been 
little  question  that  his  actions  were  totally  appropriate. 
He  could  have  recorded  that  he  dropped  the  rag  inside 
the  engine  and  was  unable  to  locate  it.  The  lead 
mechanic  was  informed  of  the  incident  and  eventually 
decided  on  his  own  that  the  rag  was  no  longer  inside 
the  engine  because  his  search  had  not  located  the  rag. 

Without  the  written  log,  it  is  difficult  to  determine  the 
actual  events  surrounding  this  incident.  The  lead 
mechanic  could  insist  that  the  mechanic  was  unsure  if 
the  rag  actually  was  inside  the  engine  or  that  he  was 
never  informed  of  the  problem,  especially  since  the 
mechanic  signed  off  on  the  repair.  Alternatively,  if  the 
problem  had  not  manifest  immediately,  the  mechanics 
involved  in  this  incident  may  then  have  been  unable  to 
provide  accurate  information  to  the  quality  assurance 
people  investigating  the  incident. 

2.  The  weekday  day  shift  and  early  part  of  the 
weekday  afternoon  shift  currently  have  far  better 
information  resources  available.  During  weekdays 
(Monday  through  Friday,  8:00  a.m.  -  5:00  p.m.),  each 
department  in  the  organization  is  fully  staffed. 
Management,  engineers,  planners,  and  the  most 
experienced  inspectors  are  all  readily  available  for 
consultation.  During  the  second  half  of  the  afternoon 
shift,  on  the  night  shift,  and  on  weekends,  it  is 
difficult  and  time-consuming  to  get  information  from 
these  resources.  For  example,  an  inspector  on  the 
weekend  shift  must  call  an  engineer  at  home  for 
consultation  on  a  technical  problem.  The  engineer,  if 
he  or  she  happens  to  be  at  home,  generally  first  tries 
to  solve  the  problem  over  the  telephone  or,  if 
appropriate,  to  postpone  addressing  the  problem  until 
the  next  weekday  shift.  The  engineer  may  be  required 
to  come  into  the  hangar  in  an  emergency,  but  this  is 
generally  the  last  resort. 

3.  Inspectors  receive  most  of  their  information, 
including  work  assignments  and  any  important  items 
from  the  previous  shift,  from  their  shift  lead. 
Therefore,  they  receive  only  information  that  the  shift 
lead  chooses  or  remembers  to  pass  along.  For 
example,  an  entry  in  the  ASRS  Database  (ACN 
#196273)  describes  the  following  incident,  which 
illustrates  potential  danger  in  filtering  critical 
information  through  the  lead  inspectors. 

Several  mechanics  noted  [that  the] 

#1  engine  [was]  making  a  loud 
unfamiliar  noise.  This  information 
was  passed  on  to  the  lead  and 
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supervisory  personnel  by  second 
shift  mechanics  so  as  to  alert  third 
shift  mechanics  who  were  to  work 
the  aircraft  that  night  and  early 
morning.  I,  the  third  shift  mechanic, 
was  assigned  to  work  this  particular 
aircraft.  However,  I  received  no 
information  concerning  this 
particular  loud  engine  noise  until 
about  ?  am  that  morning,  and  then  it 
was  passed  on  to  me  by  another 
mechanic,  not  [by]  the  lead  man  who 
assigned  me  to  work  on  the  aircraft. 

Based  upon  the  information  that  was 
made  available  to  me,  a  pilot  write¬ 
up  [of  an]  indication  problem,  [I] 
replaced  [the]  #1  engine  tac 
indicator....  Had  I  been  informed 
about  the  true  condition  of  the 
engine,  I  would  have  treated  the 
write-up  quite  differently. 

4.  Updates  to  maintenance  manuals  usually  have  a  cover  letter 
that  each  inspector  must  sign  off.  These  documents  are 
maintained  in  a  notebook  kept  in  the  inspection  office. 
Inspectors  are  expected  to  check  the  book  daily  and  to  read 
and  sign  off  any  new  entries.  This  is  easy  when  the  workload  is 
light  and  when  there  are  few  updates.  However,  when 
inspectors  are  busy  or  when  there  are  a  lot  of  updates,  many 
inspectors  fall  behind.  No  supervisor  or  lead  inspector  ever 
seems  to  question  inspectors  about  failing  to  keep  current  with 
the  updates.  An  inspector  may  learn  about  updates  only  when 
they  happen  to  relate  to  a  particular  problem  he  or  she  is 
addressing. 


user  needs  analysis  results,  we  made  the  following 
recommendations. 

1.  It  is  important  to  train  inspectors  how  to 
communicate.  Inspectors  must  learn  what  is 
expected,  so  they  understand  what  information 
must  be  communicated  and  why  it  is  important. 
Inspectors  should  be  trained  in  both  verbal  and 
written  communication  skills.  Training  also  helps 
standardize  communication  so  every  inspector  is 
able  to  pass  and  receive  useful  information. 

2.  Inspectors  must  be  challenged  to  understand  the 
importance  of  good  communication.  They  must 
understand  benefits  that  are  to  be  gained  by 
improving  communication.  Any  new 
communication  procedures  must  not  add  to 
inspectors'  workload  or  be  at  all  difficult  for 
them  to  use. 

3.  Communication  tools  must  be  developed  for  shift 
turnovers,  for  passing  general  information  such 
as  management  memos  and  aircraft  alerts,  for 
recording  detailed  problems  and  follow-ups,  as 
necessary,  etc.  The  medium  of  communication, 
e.g.,  logbook,  verbal,  blackboard,  etc.,  must  be 
chosen  that  best  meets  different  communication 
needs.  It  is  important  to  provide  only  the 
information  inspectors  need  and  not  to  overload 
them  with  unnecessary  information.  Information 
should  be  presented  in  a  form  that  is  easy  to  use 
and  that  allows  inspectors  easily  to  elicit  specific 
details,  as  necessary. 


5.  Inspectors  receive  much  information,  from  updates 
and  elsewhere,  that  they  see  as  irrelevant  to  their 
current  responsibilities.  For  example,  they  often 
receive  service  alerts  for  DC- 10s  and  Boeing  727s; 
only  DC-9  maintenance  is  performed  at  the  Atlanta 
Maintenance  Base.  Inspectors  feel  overloaded  with 
information  and  are  concerned  that  they  are  not 
always  able  to  filter  out  relevant  DC-9  information. 

6.2.S  Reselts  from  Cosuversations  with 
Manageoient 

We  also  met  with  managers  connected  with  the 
inspection  department  to  discuss  their  perceptions  of 
the  communication  system  at  Northwest  Airlines. 
Many  managers  had  never  recognized  that 
communication  problems  existed,  although  our  user 
needs  analysis  results  helped  convince  them  that  there 
was  room  for  improvement.  From  discussion  of  the 


4.  New  communication  tools  must  meet  the  needs 
and  the  expectations  of  all  involved  with  the 
inspection  department,  including  managers, 
leads,  and  inspectors.  These  individuals  need  to 
have  input  into  redesigning  the  communication 
system. 

5.  At  the  Atlanta  Maintenance  Base,  there  are  three 
distinct  inspection  groups:  support  shops,  engine 
shops,  and  major  maintenance.  The 
communication  system,  especially  the  shift 
turnover  log,  should  be  standardized  for  all  these 
groups.  Such  standardization  would  make  it 
easier  for  inspectors  to  move  among  groups, 
effectively  obtaining  necessary  information,  and 
allowing  better,  more-effective  cross-utilization 
of  personnel. 

6.  The  maintenance  department  holds  a  daily  8:00 
a.m.  production  meeting;  the  inspection 
department  is  invited  to  attend  this  meeting.  The 
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information  from  this  meeting  should  be  used  to 
help  schedule  tasks  for  the  afternoon  and  night 
shifts.  The  day  shift  attendee  at  this  meeting  must 
relay  information  through  a  shift  turnover  log  to 
the  other  shifts.  It  should  become  standard 
practice  to  use  the  shift  turnover  log  to 
communicate  such  information. 


After  v^e  completed  the  broad-based  user  needs 
analysis  of  workers  and  management,  we 
considered  possible  solutions  for  improved 
communication  at  the  maintenance  base. 

6.3.1  Communication  Tools 

As  discussed  above,  communication  could  be 
facilitated  by  implementing  a  new  communication 
system.  However,  in  choosing  the  most  appropriate 
tool  for  improving  communication,  it  is  necessary  to 
consider  who  is  trying  to  communicate  with  whom 
and  what  is  being  communicated.  The  human  factors 
principle  of  fitting  the  tools  to  the  user  applies  here 
no  less  than  in  designing  hand  tools.  It  may  be 
necessary  to  use  different  communication  tools  to 
satisfy  different  types  of  communication 
requirements;  in  fact,  it  is  improbable  that  one 
communication  tool  could  address  all  communication 
needs. 

6.3.1. 1  Available  Communication  Tools 

A  formal  written  log,  e.g.,  the  shift  turnover  log,  is  a 
permanent  written  record  of  activities  within  the 
inspection  department.  The  document  can  serve 
legally  as  evidence  for  scheduling/staffing 
considerations  and  job  control,  and  as  a  written 
account  of  problems  inspectors  encountered.  A 
formal  written  log  is  usually  bound  so  that  pages  and 
the  information  on  them  cannot  be  removed. 

Informal  written  notes  can  substitute  for  the  current 
reliance  on  memory  and  verbal  communication. 
Inspectors  may  forget  to  pass  on  information  to  the 
lead  inspector  or  to  inspectors  on  the  next  shift. 
Writing  down  information  relieves  the  inspector  of 
relying  on  memory  for  the  transfer  of  information. 
Informal  notes  can  be  addressed  to  an  individual  or  to 
an  entire  crew. 

Tape  recorders  can  replace  informal  written  notes 
(discussed  above).  Many  inspectors  do  not  like  to 
write  down  information  because  the  process  of  doing 


so  is  cumbersome  and  time-consuming.  Allowing 
each  inspector  to  make  personal  notes  and  notes  to 
others  on  a  tape  recorder  eliminates  the  need  for 
written  notes.  The  tape  can  then  be  transcribed  into  a 
written  log  and/or  passed  to  the  oncoming  shift  for 
the  next  inspector.  This  allows  an  inspector  to  replay 
verbal  information  during  a  shift.  Tape  recorders  are 
best  suited  for  recording  information  for  self¬ 
reminding  or  for  another  individual  in  a  closely 
related  occupation. 

Computer  software  tools  can  be  developed  to  meet 
inspectors'  communication  needs.  Tools  such  as 
electronic  mail,  electronic  bulletin  boards,  electronic 
turnover  logs,  electronic  databases,  etc.,  can  transfer 
information  among  people.  A  computer  tool  allows 
more  than  one  person  to  access  information 
simultaneously;  this  is  not  feasible  with  a  formal 
written  log  since  there  is  only  one  copy.  Electronic 
tools  provide  flexibility  in  the  presentation  of 
information.  For  example,  each  inspector  may  request 
only  information  directly  pertaining  to  the  task  at 
hand,  and  the  inspector  will  not  have  to  read 
irrelevant  information  (see  comments  in  #5  of  Section 
6.2.4). 

Blackboards/Whiteboards  are  quite  useful  for 
recording  information  that  only  needs  to  be  used  for  a 
short  time.  Blackboards/Whiteboards  should  be 
utilized  for  communicating  information  to  an  entire 
crew  since  the  information  becomes  general 
knowledge.  Information  could  be  left  on  the  board  for 
each  of  the  three  shifts  to  see  and  then  be  erased.  It  is 
important  not  to  erase  information  that  might  be 
needed  later,  unless  it  is  transcribed  into  a  permanent 
written  log.  For  example,  inspector  work  assignments 
are  generally  written  on  a  whiteboard  during  every 
shift.  This  board  is  erased  at  the  end  of  every  shift, 
and  work  assignments  are  not  recorded.  It  is  therefore 
difficult  quickly  to  trace  previous  work  assignments; 
one  must  research  completed  workcards  to  do  so. 

Formal  crew  meetings  are  useful  for  presenting 
information  to  all  inspectors.  Meetings  permit  two- 
way  discussions  about  the  information,  as  well  as  the 
opportunities  for  questions.  Since  the  same 
information  can  be  presented  to  all  three  shifts,  this 
ensures  that  all  inspectors  receive  the  same 
information.  However,  crew  meetings  are  often 
ineffective  in  meeting  inspectors'  communication 
needs.  Inspectors  often  ask  questions  at  these 
meetings  that  are  never  answered,  and  the  meetings 
can  turn  into  gripe  sessions. 
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Although  loforma!  verba!  coiBmonication  is  used  in 
many  information  exchanges,  it  is  not  well-suited  for 
many  tasks.  Verbal  communication  is  short-lived.  If 
the  person  receiving  verbal  information  forgets 
something,  it  is  very  difficult  for  his  or  her  memory  to 
be  refreshed.  An  inspector  could  be  in  the  position  of 
having  to  call  an  off-duty  inspector  at  home  to  have 
information  repeated.  On  the  other  hand,  an  inspector 
may  refer  to  a  written  record  of  information  as  many 
times  as  necessary.  Thus,  written  communication  is 
less  demanding  on  an  inspector’s  memory.  In 
addition,  relying  on  memory  for  recording 
information  is  ineffective  if  the  information  needs  to 
be  kept  for  a  long  time.  For  example,  an  inspector 
who  discovers  and  resolves  a  particular  problem  on 
an  aircraft  may  not  recall  details  of  what  occurred 
five  months  later,  when  the  FAA  is  questioning  him 
or  her  about  a  critical  incident  with  that  aircraft. 
Generally,  verbal  communication  to  more  than  one 
individual  is  difficult  because  it  is  nearly  impossible 
to  relay  verbally  exactly  the  same  information,  in 
exactly  the  same  manner,  more  than  once. 

Inspectors  use  noe-roiatsirfie  workcards  (NR  W/Cs) 
to  identify  areas  on  an  aircraft  that  require 
maintenance.  The  workcards  are  a  formal  recording 
procedure  that  allows  inspectors  to  communicate  their 
findings  to  the  mechanics  who  will  perform  the 
needed  repairs.  Each  non-routine  workcard  is  then 
bought  back  to  the  inspector,  who  rereads  the  original 


write-up  to  ensure  that  the  work  is  completed  as 
specified. 

Table  6.3  (next  page)  illustrates  how  various  tools 
can  be  used  to  meet  communication  needs  between 
various  inspection  and  maintenance  personnel. 

As  ASRS  report  analysis  indicates,  the  issue  in 
choosing  an  appropriate  communication  tool  is  one  of 
ensuring  ease  of  use  so  that  necessary  communication 
occurs.  Table  6.3  (next  page)  shows  a  matrix  of 
which  tools  can  be  useful  for  which  tasks.  For 
example,  a  small  tape  recorder,  such  as  a  micro¬ 
cassette  dictating  machine,  provides  easy  and  rapid 
memory  augmentation.  In  some  organizations, 
inspectors  have  such  a  device  taped  to  their  flashlight 
so  as  to  have  it  instantly  accessible.  This  is  an 
example  of  improving  ease  of  use  and,  hence,  of 
decreasing  the  probability  of  missed  communication. 

Another  example  is  a  board  which  can  be  used  for 
rapid  communication  with  many  people.  Although 
Table  6.3  (next  page)  indicates  that  a  board  can  be 
used  by  leads  and  managers,  it  can  also  serve  as  a 
source  of  situational  awareness  when  it  carries  notes 
from  inspectors  or  mechanics.  Again,  the  primary 
function  of  this  tool  is  to  promote  ease  of  use. 
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manager  to  crew 
(al!  shifts) 


mechanic  to  lead 

inspector _ 

mechanic  to  inspector 


As  Table  6.3  shows,  computer  systems  are  available 
to  facilitate  almost  any  activity,  but  their  ease  of  use 
is  not  always  appropriate  for  the  demands  of 
communication.  If  people  need  to  be  trained  and  then 
must  later  remember  how  to  access  the  tool,  or  how  to 
direct  a  notice,  then  the  tool’s  frequency  of  use  will 
drop.  Fortunately,  advances  in  human-computer 
interaction  (HCI)  have  improved  interface  design, 
particularly  for  infrequent  users. 

The  other  major  cluster  of  tool  use  is  in  handwritten 
logs.  The  shift  turnover  log  is  the  basis  for  human 
factors  intervention  in  this  project. 


6.3.2  Proposed  Shift  Turnover  Log 

The  proposed  shift  turnover  log  was  designed  to 
improve  communication  among  inspectors  from 
different  shifts.  The  present  shift  turnover  log  is  used 
mainly  by  the  lead  inspectors  and  does  not  contain 
much  information  that  inspectors  can  utilize.  It  does 
not  lecord  activities  that  took  place  during  a  shift  or 
help  the  next  shift  know  what  they  need  to 
accomplish. 

The  proposed  shift  turnover  log  is  intended  for  use  by 
all  inspectors.  It  allows  an  inspector  to  record 
activities  during  a  shift,  leaving  a  written  account  of 
what  needs  to  be  accomplished  and  helping  prevent 
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rework.  Rework  in  inspection,  i.e.,  more  than  one 
inspection  of  the  same  area,  is  often  caused  by 
miscommunication  between  two  inspectors.  This  is 
especially  true  when  an  inspection  is  carried  over 
from  one  shift  to  the  next,  and  the  second  inspector 
does  not  understand  where  to  start  and  stop  the 
inspection.  In  this  situation,  an  inspector  typically 
does  "a  bit  more”  so  there  is  no  doubt  the  workcard 
was  covered. 

6.3.2.1  First  Draft  General  Information:  Proposed  Shift 
Turnover  Log 

This  proposed  shift  turnover  log  (Figure  6.1,  next 
page)  will  allow  inspectors  easily  to  obtain  necessary 
information  about  an  aircraft  to  which  they  are 
assigned.  This  log  is  organized  into  five  separate, 
bound  books.  Each  book  has  sequentially  numbered 
pages  to  prevent  any  pages  from  being  removed. 

The  first  book  is  the  general  shift  turnover  log.  It  can 
be  used,  as  the  current  log  is  used,  to  pass  information 
between  shift  leads. 


Information  included  in  this  log  includes  any 
personnel  information  such  as  assigned  overtime, 
call-ins,  and  field-trips,  as  well  as  any  general 
problems.  The  shift  lead  inspector  should  complete 
this  log  for  the  following  shift. 

The  other  four  logs  correspond  to  the  hangar  bays 
(Figure  6.2,  page  72).  Each  book,  including  the 
pages,  is  color-coded  to  match  the  bay  color.  The 
book  should  contain  enough  pages  for  it  to  be  used 
during  the  estimated  duration  of  the  aircraft's  stay  in 
the  hangar:  three  pages  for  each  day,  plus  a  few  extra. 
A  new  book  can  be  started  for  each  new  aircraft; 
therefore,  each  book  contains  the  complete  inspection 
history  for  one  aircraft.  The  log  can  be  filed  when  the 
aircraft  leaves  the  hangar.  Inspectors  assigned  to  a 
particular  aircraft  should  complete  this  log. 

The  specifications  and  instructions  for  the  proposed 
shift  turnover  log  are  included  as  Appendix  6-D. 
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6.3.2.2  Evaluation  of  First  Draft 
A  sample  of  the  inspectors  was  asked  to  evaluate  the 
proposed  shift  turnover  log.  Responses  of  the 
seventeen  inspectors  are  summarized  in  Table  6.4. 

These  results  indicate  that  the  proposed  shift  turnover 
log  offers  many  improvements  over  the  current 
version.  A  One-Sample  Wilcoxon  test  was  performed 
to  determine  whether  the  median  response  for  each 
question  was  significantly  different  from  the  0,  mid- 
point(4),  or  end-point  of  the  rating  scale(8).  After 
performing  this  analysis,  we  find  that  the  inspectors 
felt  that  the  use  of  a  separate  log  for  recording 
personnel  issues  and  general  problems  was 
significantly  better  than  useful  (median  =  5.65, 


p=.038).  They  also  indicated  that  they  would  read  the 
turnover  log  for  the  aircraft  to  which  they  were 
assigned  more  than  three  times  per  week  (median  = 
7.0,  p=.009).  Inspectors  also  felt  that  the  proposed 
turnover  log  was  more  useful  than  the  current 
turnover  log  (median  =  5.225,  p=.002)  and  that  they 
would  use  the  proposed  turnover  log  more  often  than 
they  use  the  current  turnover  log  (median  =  4.5, 
p=.037)- 

Other  trends  in  the  data,  although  not  statistically 
significant,  are  that  the  inspectors  generally  found  the 
proposed  log  easy  to  understand  and  that  both  the 
general  and  the  aircraft  sections  contain  the  right 
amount  of  information.  Unfortunately,  inspectors 


User  Needs  Analysis  Question 

Average 

Std.  Deviation 

How  useful  is  a  separate  log  (for  lead  inspectors)  for  personnel  information  and 
general  problems? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

5.44 

2.49 

How  useful  is  a  separate  log  for  each  hangar  bay? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.09 

2.72 

How  useful  is  the  practice  of  maintaining  a  separate  log  for  each  aircraft? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.88 

2.5 

Rate  the  ease  of  understanding  of  the  proposed  shift  turnover  log: 

0  -  Not  At  All  Easy  4  -  Easy  8  -  Very  Easy 

4.53 

2.18 

Rate  the  usefulness  of  the  information  in  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.24 

2.14 

How  often  would  you  read  all  sections  of  the  proposed  turnover  log? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

4.63 

2.8 

How  often  would  you  read  the  section  of  the  log  for  the  aircraft  that  you  are 
assigned  to? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

6.33 

2.54 

How  often  would  you  make  an  entry  into  the  turnover  log? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

4.21 

2.93 

Rate  the  amount  of  information  in  the  general  section  of  the  proposed  turnover 
log: 

0  -  Not  Enough  Info.  4  -  Right  Amt.  of  Info.  8  -Too  Much  Info. 

4.09 

1.85 

Rate  the  amount  of  Information  in  the  aircraft  section  of  the  proposed  turnover 
log: 

0  -  Not  Enough  Info.  4  -  Right  Amt.  of  Info.  8  -  Too  Much  Info. 

4.29 

1.99 

Rate  the  type  of  information  in  the  general  section  of  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.81 

1.78 

Rate  the  type  of  information  in  the  aircraft  section  of  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.83 

1.85 

How  does  the  proposed  turnover  log  compare  to  the  current  turnover  log? 

0  -  Less  Useful  4  -  As  Useful  8  -  More  Useful 

5.38 

1.51 

How  often  would  you  use  the  proposed  log,  as  compared  to  your  use  of  the 
current  log? 

0  -  Sig.  Less  4  -  About  the  Same  8  -  Sig.  More 

4.85 

1.61 

How  do  you  like  the  format  of  the  general  section  of  the  proposed  turnover  log? 

0  -  Not  Easy  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

3.91 

1.11 

How  do  you  like  the  format  of  the  aircraft  section  of  the  proposed  turnover  log? 

0  -  Not  Easy  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

3.64 

1.31 

How  useful  is  the  current  shift  turnover  log? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.35 

1.63 

How  useful  is  the  proposed  shift  turnover  log? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.64 

1,38 
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indicated  that  they  would  be  likely  to  make  an  entry 
in  the  log  only  three  times  per  week,  not  every  day  as 
the  log  would  require.  Comments  from  the  user  needs 
analysis  indicated  that  many  inspectors  feel  that 
maintaining  the  log  is  the  lead  inspector’s  duty.  There 
are  clear  issues  of  culture,  expectations,  and  training 
surrounding  any  change  in  the  shift  turnover  log. 

The  inspectors  indicated  that  the  proposed  shift 
turnover  log  does  not  meet  their  needs  for 
information,  as  indicated  by  the  less-than-useful 
ratings  given  to  the  type  of  information  the  log 
contains.  They  do  not  find  the  proposed  shift  turnover 
log's  layout  particularly  easy  to  use.  Finally, 
inspectors  rated  the  usefulness  of  the  proposed  shift 
turnover  log  (Questions  17  and  18:  mean  4.64 
compared  to  4.35)  as  only  slightly  higher  than  the 
usefulness  of  the  current  shift  turnover  log;  a  Mann- 
Whitney  analysis  indicates  that  this  difference  is  not 
statistically  significant. 

6.33  Version  2  of  the  Shift  Change  Log 

6.3.3.1  Design  of  Second  Version  of  Shift  Change  Log 

From  these  results,  it  appears  that  inspectors  approve 
of  the  idea  of  developing  a  new  format  for  the  shift 
turnover  log  and  will  utilize  an  improved  log, 
especially  its  sections  pertaining  to  their  specific 
work  assignments.  However,  more  work  is  necessary 
to  find  a  layout  that  will  meet  inspectors'  information 
needs. 

After  analyzing  the  results,  we  concluded  that 
inspectors  supported  the  idea  of  maintaining  a 
separate  log  for  each  hangar  bay;  however,  they  were 
not  satisfied  with  the  information  on  or  the  format  of 
the  proposed  log.  More  work  was  needed  to  design  a 
log  better  meeting  the  inspectors’  information  needs. 
We  decided  to  use  a  team  approach  for  the  next  phase 
of  shift  turnover  log  design.  We  held  meetings  with 
each  inspection  shift  to  discuss  how  the  log  should  be 
designed.  Inspectors  were  encouraged  to  contribute  to 
the  process  by  indicating  the  information  they  would 
like  to  see  included  in  the  turnover  log. 

Unfortunately,  of  the  10  to  15  inspectors  in  each 
meeting,  only  a  few  provided  input  for  redesigning 
the  shift  turnover  log.  Their  overall  suggestions  were 
to  simplify  the  proposed  shift  turnover  log  and  to 
reduce  the  writing  required  to  complete  it.  One 
inspector  suggested  that  the  log  should  include  only  a 
simple  heading  (aircraft  number,  date,  shift)  and  a 
blank  space  for  inspectors  to  write;  this  is  basically 


the  same  as  the  current  turnover  log  (it  is  not  being 
utilized  effectively). 

Although  user  needs  analysis  results  had  indicated 
otherwise,  most  inspectors  reacted  negatively  to  the 
idea  of  a  redesigned  turnover  log.  Some  of  their 
opinions  were  the  following:  1)  inspectors  would  not 
use  a  redesigned  log  unless  it  was  mandated  by  upper 
management;  2)  separating  the  log  by  hangar  bay 
would  make  the  log  too  difficult  for  leads  to  use;  3) 
leads  are  the  only  ones  who  need  a  shift  turnover  log; 
4)  inspectors  depend  on  leads  to  pass  along 
information;  and  5)  it  is  not  the  inspectors’ 
responsibility  to  pass  information  during  a  shift 
turnover.  These  comments  were  symptomatic  of 
inspectors’  general  attitudes,  implying  that 
communication  between  shifts  is  not  the  most  serious 
problem  within  the  inspection  department. 

In  addition,  the  shift  schedule  (7:00  a.m.-3:00  p.m., 
3:00  p.m.-l  1:00  p.m.,  and  1 1:00  p.m.-7:00  a.m.) 
does  not  allow  for  overlap  of  oncoming  and  outgoing 
shifts.  Many  inspectors  felt  that  a  shift  turnover  log 
(either  verbally  or  written)  would  require  too  much 
time  and  would  place  too  many  additional 
requirements  on  the  inspectors.  What  the  inspectors 
fail  to  realize  is  that  this  is  the  exact  reason  an 
effective  shift  turnover  log  is  essential. 

Inspectors  also  indicated  that  it  is  the  lead  inspector’s 
responsibility  to  perform  a  shift  turnover.  The  lead 
should  extract  the  important  information  from  each 
crew  member  and  pass  this  information  to  the  next 
shift.  The  oncoming  lead  is  responsible  for  reading 
the  information  in  the  log  and  distributing  it,  as 
necessary.  Although  many  inspectors  indicated  that 
they  require  information  passed  between  shifts,  they 
believe  that  someone  else  is  responsible  for  providing 
this  information. 

Many  inspectors  indicated  that  they  would  find  a  log 
for  the  particular  aircraft  to  which  they  were  assigned 
helpful.  This  would  allow  them  quickly  to  ‘get  a  feel’ 
for  the  aircraft’s  status.  These  inspectors  also  stated 
that  it  is  most  important  for  leads  to  understand  what 
is  happening,  and  the  proposed  shift  turnover  log 
should  be  designed  for  leads,  not  for  other  crew 
members.  This  is  troubling;  as  one  sees  in  the  ASRS 
reports,  it  is  critical  for  inspectors  working  on  an 
aircraft  to  have  a  good  understanding  of  the  problems 
previous  shifts  encountered. 

In  addition,  many  inspectors  have  regular 
opportunities  to  serve  as  the  lead  for  a  shift,  e.g., 
when  the  permanent  lead  takes  a  day  off,  and  many 
inspectors  eventually  become  permanent  leads. 
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Although  inspectors  do  not  feel  responsible  for 
knowing  information  in  the  turnover  log,  they  are 
expected  to  have  a  full  understanding  of  it  when  they 
act  as  lead  for  a  shift.  An  effective  turnover  log  could 
ensure  that  an  acting  lead  inspector  is  quickly  able  to 
extract  necessary  information.  If  all  inspectors 
regularly  read  the  redesigned  log,  there  will  be  less 
information  to  absorb  when  he  or  she  becomes  a 
temporary  lead  inspector. 

There  also  seems  to  be  a  large  mismatch  between  the 
inspectors’  need  for  information  and  the  effort  they 
are  willing  to  make  to  obtain  it.  On  the  original 
communications  user  needs  analysis,  inspectors 
indicated  that  they  rarely  if  ever  have  enough 
information,  that  they  often  must  search  for 
information  to  perform  their  jobs,  and  that  they  would 
like  information  to  be  readily  available.  However, 
when  inspectors  were  asked  to  provide  more 
information  about  events  occurring  during  their  shift 
through  the  shift  turnover  log,  most  were  extremely 
reluctant  to  do  so.  They  felt  that  completing  a  written 
log  at  the  end  of  each  shift  would  be  too  time- 
consuming  and  difficult.  Inspectors  seem  to  want  to 
receive  information  from  the  previous  shifts,  but  not 
to  provide  information  to  the  next  shift. 

Inspectors  are  reluctant  to  write  down  any 
information  not  specifically  required.  They  feel  that 
their  signatures  on  workcards  fulfill  their  legal  record 
keeping  requirements.  They  do  not  want  to  record 
additional  information  in  a  log  which  could  be  used 
against  them  in  an  investigation;  they  do  not  realize 
that  information  in  a  written  log  could  protect  them  in 
an  investigation.  This  is  also  part  of  a  current  national 
debate:  can  maintenance  and  inspection  personnel  be 


disciplined  merely  for  providing  information  which 
could  help  the  system? 

Many  inspectors  seem  unwilling  to  make  an  effort  to 
improve  the  communication  process.  They  are 
unhappy  with  how  management  treats  them  and,  thus, 
have  little  motivation  to  improve  the  situation.  Most 
simply  want  to  perform  their  jobs  and  to  take  on  as 
little  responsibility  as  possible.  Inspectors  are 
distrustful  of  management  and  do  not  believe  that 
management  wants  to  aid  the  inspectors  by  trying  to 
improve  communication.  During  small  group  (or  one- 
on-one)  discussions,  inspectors  offered  suggestions 
for  improving  internal  communication  in  the 
inspection  department.  During  the  shift  meetings  few 
people  were  willing  to  discuss  a  need  for  improved 
communication.  Even  individual  inspectors  who  want 
to  improve  their  jobs  do  not  want  to  appear 
sympathetic  to  management’s  needs  or  wants.  Some 
inspectors  had  a  hard  time  believing  that  management 
had  not  sent  us.  Sociotechnical  problems  between 
management  and  inspectors  must  be  resolved  before 
any  proposed  shift  turnover  log  can  meet  information 
needs  of  both  groups.  As  is  true  of  many  human 
factors  issues  in  aircraft  maintenance  and  inspection, 
searching  for  a  consensus  solution  to  a  technical 
problem  reveals  broad  social  issues  when  it  is  time 
for  implementation. 

Based  on  input  we  received  in  evaluation  meetings, 
we  simplified  the  shift  change  log  for  its  final  version. 
We  did  this  to  address  inspectors’  (other  than  leads’) 
unwillingness  to  provide  shift  information,  although 
the  changes  somewhat  reduce  the  information’s  utility 
to  the  reader.  Figures  6.3  and  6.4  (following  pages) 
show  the  second  draft  of  the  shift  change  log. 
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Date:  To  Be  Read  By:  Morning  Afternoon  Night  Shift 

Lead  Inspector: 

Manager: 

Filled  In  By:  on  the  Morning  Afternoon  Night  Shift 

Personnel  Information 

Call-Ins 

Name 

Reason 

Time 

1  Overtime  1 

Name 

Reason 

Number  of  Hours 

1  Field  Trips  1 

Name 

Destination 

Departure  Time 

Return 

Time 

Special  Instructions/General  Problems 

Problem 

Needed  Action/Alert 
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6.3.3.2  Evaluation  of  Version  2  of  the  Shift  Change 
Log 

We  used  the  same  evaluation  form  as  in  Section 

63.2.2  to  obtain  feedback  on  Version  2  of  the  new 
shift  change  log.  Nineteen  inspectors  evaluated  the 
log  shown  in  Figures  6.3  (previous  page)  and  6.4. 
Table  6.5  (next  page)  summarizes  these  results  in  the 
same  way  Table  6.4  summarized  those  for  the  first 
version. 

A  One-Sample  Wilcoxon  test  showed  that  inspectors 
still  appreciated  the  idea  of  separating  personnel 
information  from  aircraft  information  (median  = 
5.025,  p  =  .01 1),  that  they  found  information  in  the 
proposed  log  more  than  useful  (median  4.95),  p  = 
.003),  that  they  would  read  all  sections  of  the  log 


more  than  three  times  per  week  (median  =  5.300,  p  - 
.036),  that  they  would  read  the  section  of  the  log  for 
the  aircraft  to  which  they  were  assigned  almost  every 
shift  (median  =  7.375,  p  =  .001),  and  that  they  would 
make  entries  into  the  log  more  than  three  times  per 
week  (median  =  6.00,  p  ==  .023). 

Inspectors  also  thought  that  information  in  the  log’s 
general  section  is  more  than  useful  (median  =  4.562, 
p  =  .015),  and  that  information  in  the  aircraft  section 
is  more  than  useful  (median  =  4.600,  p  -  .012).  They 
preferred  the  proposed  to  the  current  turnover  log 
(median  =  5.450,  p  .001)  and  would  use  the 
proposed  log  more  than  they  use  the  current  log 
(median  =  5.150,  p  =  .005).  Inspectors  found  the 
new  format  of  both  general  and  aircraft  sections 
better  than  easy  to  use  (median  ==  4.650, 4.738,  p  = 
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User  Analysis' Question .  :  : 

Average 

Std;  Deviation 

How  useful  is  a  separate  log  (for  lead  inspectors)  for  personnel  information  and 
general  problems? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

5.08 

1.56 

How  useful  is  a  separate  log  for  each  hangar  bay? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.09 

2.10 

How  useful  is  the  practice  of  maintaining  a  separate  log  for  each  aircraft? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.50 

2.27 

Rate  the  ease  of  understanding  of  the  proposed  shift  turnover  log: 

0  -  Not  At  All  Easy  4  -  Easy  8  -  Very  Easy 

4.70 

1.69 

Rate  the  usefulness  of  the  information  in  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

5.05 

1.34 

How  often  would  you  read  all  sections  of  the  proposed  turnover  log? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

5.35 

2.57 

How  often  would  you  read  the  section  of  the  log  for  the  aircraft  that  you  are 
assigned  to? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

6.98 

1.64 

How  often  would  you  make  an  entry  into  the  turnover  log? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

5.96 

2.19 

Rate  the  amount  of  information  in  the  general  section  of  the  proposed  turnover 
log: 

0  -  Not  Enough  info.  4  -  Right  Amt.  of  Info.  8  -Too  Much  Info. 

4.16 

1.01 

Rate  the  amount  of  information  in  the  aircraft  section  of  the  proposed  turnover 
log: 

0  -  Not  Enough  Info.  4  -  Right  Amt.  of  Info.  8  -  Too  Much  Info. 

4.14 

1.02 

Rate  the  type  of  information  in  the  general  section  of  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.77 

1.25 

Rate  the  type  of  information  in  the  aircraft  section  of  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.83 

1.29 

How  does  the  proposed  turnover  log  compare  to  the  current  turnover  log? 

0  -  Less  Useful  4  -  As  Useful  8  -  More  Useful 

5.48 

1.42 

How  often  would  you  use  the  proposed  log,  as  compared  to  your  use  of  the 
current  log? 

0  -  Siq.  Less  4  -  About  the  Same  8  -  Sig.  More 

5.20 

1.51 

How  do  you  like  the  format  of  the  general  section  of  the  proposed  turnover  log? 

0  -  Not  Easy  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

4.86 

1.49 

How  do  you  like  the  format  of  the  aircraft  section  of  the  proposed  turnover  log? 

0  -  Not  Easy  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

4.93 

1.52 

How  useful  is  the  current  shift  turnover  log? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.12 

1.69 

How  useful  is  the  proposed  shift  turnover  tog? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

5.26 

1.43 

.015,  .016).  Finally,  they  indicated  that  the  proposed 
log  is  more  than  useful  (median  =  5.200,  p  =  ,002). 

It  is  possible  to  use  data  in  Tables  6.4  (previous  page) 
and  6.5  directly  to  compare  the  two  versions  of  the 
shift  change  log.  A  two-sample  turnover  test  was 
performed  to  compare  results  from  the  evaluations  of 
the  first  and  second  drafts.  Table  6.6  (next  page) 
presents  the  results  of  this  analysis. 

These  results  indicate  that  inspectors  rated  the  second 
draft  significantly  higher  in  both  information  content 
and  format  (at  the  p  <  .01  significance  level).  Since 
these  were  the  first  draft’s  main  weaknesses,  the 
second  draft  appears  better  able  to  meet  inspectors’ 
communication  needs. 


Although  the  result  was  not  significant,  inspectors  felt 
that  the  second  draft  was  more  useful  (mean  ==  5.26 
versus  4.64  in  first  draft)  and  that  they  would  be  more 
likely  to  make  frequent  entries  in  the  second  draft 
(mean  =  5.96  versus  4.21).  These  data  support  the 
findings  that  the  second  draft  is  better  suited  to 
inspectors’  communication  needs.  We  therefore 
proposed  that  this  version  become  the  base’s  standard 
shift  change  log. 

6.3.4  Other  Communication  Solutions 

During  1995,  Northwest  Airlines  management  will 
implement  two  programs  to  improve  communication 
with  its  workforce.  First,  they  will  introduce  a  bulletin 
board  for  posting  company  news  and  announcements. 
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User  Needs  Analysis  Question 

1st  Draft 
Mean 

2nd  Draft 
Mean 

P  Value 

How  useful  is  a  separate  log  (for  lead  inspectors)  for  personnel  information 
and  general  problems? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

5.44 

5.08 

0.61 

How  useful  is  a  separate  log  for  each  hangar  bay? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.09 

4.09 

1.0 

How  useful  is  the  practice  of  maintaining  a  separate  log  for  each  aircraft? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.88 

3.50 

0.64 

Rate  the  ease  of  understanding  of  the  proposed  shift  turnover  log; 

0- Not  At  All  Easy  4 -Easy  8 -Very  Easy 

4.53 

4.70 

0.79 

Rate  the  usefulness  of  the  information  in  the  proposed  turnover  log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.24 

5.05 

0.19 

How  often  would  you  read  all  sections  of  the  proposed  turnover  log? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

4.63 

5.35 

0.43 

How  often  would  you  read  the  section  of  the  log  for  the  aircraft  that  you  are 
assigned  to? 

0  -  Never  4  -  3  times/week  8  -  Every  Shift 

6.33 

6.98 

0.40 

How  often  would  you  make  an  entry  into  the  turnover  log? 

0  .  Never  4  -  3  times/week  8  -  Every  Shift 

4.21 

5.96 

0.11 

Rate  the  amount  of  information  in  the  general  section  of  the  proposed 
turnover  log: 

0  -  Not  Enouah  Info.  4  -  Right  Amt.  of  Info.  8  -Too  Much  Info. 

4.09 

4.16 

0.90 

Rate  the  amount  of  information  in  the  aircraft  section  of  the  proposed  turnover 
log: 

0  -  Not  Enough  Info.  4  -  Right  Amt.  of  Info.  8  -  Too  Much  Info. 

4.29 

4.14 

0.79 

Rate  the  type  of  information  in  the  general  section  of  the  proposed  turnover 
log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.81 

4.77 

0.081 

Rate  the  type  of  information  in  the  aircraft  section  of  the  proposed  turnover 
log: 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

3.83 

4.83 

0.081 

How  does  the  proposed  turnover  log  compare  to  the  current  turnover  log? 

0  -  Less  Useful  4  -  As  Useful  8  -  More  Useful 

5.38 

5.48 

0.85 

How  often  would  you  use  the  proposed  log,  as  compared  to  your  use  of  the 
current  log? 

0  -  Sia.  Less  4  -  About  the  Same  8  -  Sig.  More 

4.85 

5.20 

0.51 

How  do  you  like  the  format  of  the  general  section  of  the  proposed  turnover 
log? 

0  -  Not  Easv  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

3.91 

4.86 

0.038 

How  do  you  like  the  format  of  fhe  aircraft  section  of  the  proposed  turnover 
log? 

0  -  Not  Easv  To  Use  4  -  Easy  To  Use  8  -  Very  Easy  To  Use 

3.64 

4.93 

0.011 

How  useful  is  the  current  shift  turnover  log? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.35 

4.12 

0.68 

How  useful  is  the  proposed  shift  turnover  log? 

0  -  Of  No  Use  4  -  Useful  8  -  Extremely  Useful 

4.64 

5.26 

0.19 

Each  shift  will  have  its  own  copy  of  each 
announcement,  and  each  inspector  will  sign  off 
after  reading  each  posting.  This  system  is  designed  to 
ensure  that  all  inspectors  are  aware  of  important 
company  business. 

Management  will  also  schedule  meetings  with 
inspectors,  and  inspectors  will  determine  the 
frequency  of  these  meetings.  These  meetings  will  help 
management  better  understand  each  inspector's  needs 
and  concerns.  Inspectors  issues  and  concerns  will  be 


recorded  on  a  form  that  includes  to  whom  the  issue  is 
assigned  and  an  expected  resolution  date.  The  form 
will  be  posted  on  the  bulletin  board  so  that  everyone 
is  aware  of  progress  made  toward  resolving  the 
issues. 

Other  possible  solutions  inspectors  suggested  include 
the  following. 

1 .  Allow  each  inspector  to  carry  a  small  tape 

recorder  throughout  the  day  so  that  an  inspector 
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can  record  information,  notes,  and  messages  as 
events  happen.  The  tapes  can  be  passed  to  the 
inspector  taking  over  on  the  next  shift.  This 
second  inspector  can  listen  to  the  previous 
inspector’s  notes  as  often  as  necessary.  The  tapes 
can  be  transcribed  into  the  written  log  of  daily 
activities  for  permanent  record  keeping. 

2.  Develop  a  shift  turnover  log  in  the  form  of  a 
simple  checklist,  allowing  inspectors  quickly  to 
complete  the  log  with  minimal  writing. 
Eventually,  a  bar  code  system  could  allow  even 
simpler  completion. 

3.  Use  one-on-one  shift  turnovers  in  which 
incoming  inspectors  walk  around  the  hangar  with 
outgoing  inspectors  to  ensure  that  all  necessary 
information  is  relayed. 

4.  Use  a  blackboard/whiteboard  temporarily  to 
record  information  that  may  be  useful  for  all 
inspectors.  Information  often  passes  to  inspectors 
through  informal,  impromptu  meetings,  often 
over  a  particular  problem  one  inspector 
encountered.  When  absent,  a  particular  inspector 
may  never  know  that  he  or  she  missed  hearing 
important  information.  When  this  problem  is 
again  encountered,  it  may  be  completely  new  to 
some  inspectors,  although  others  previously 
discussed  and  resolved  it.  Inspectors  would  find 
it  helpful  for  this  type  of  information  to  be 
written  down  so  that  they  all  may  review  it. 


One  of  the  outcomes  of  this  study  was  to  be  a 
guide  for  airlines  on  how  to  establish  and 
implement  their  own  human  factors/ergonomics 
programs.  The  information  on  task  force  formation, 
training,  and  procedures  was  written  as  a  guide  in 
Chapter  2  of  the  FAA's  Human  Factors  Guide  for 
Aviation  Maintenance. 

That  chapter  presents  the  following  seven-step 
process: 


This  material  was  presented  and  used  as  the  basis  for 
a  workshop  at  the  FAA/AAM  Annual  Human  Factors 
in  Maintenance  meeting  in  Albuquerque,  New 
Mexico,  during  November  1994.  C.  G.  Drury 
summarized  progress  of  the  current  project  in  a 
presentation  entitled  "Integrating  Human  Factors  into 
Maintenance  Program."  Project  results  since  that  time 
(Sections  3  and  4  of  this  report)  provide  additional 
feasible  structures  for  human  factors  implementation. 
A  broader  program  with  limited  objectives,  but  wide 
involvement,  may  serve  as  a  viable  first  project  to 
gain  visibility  for  human  factors  in  a  maintenance 
organization.  Lessons  learned  from  the 
communications/shift  log  study  reported  in  Sections  3 
and  4  are  being  incorporated  into  Chapter  2  of  the 
Guide  and  will  form  the  basis  of  a  proposed  new 
Guide  chapter  covering  communications  processes. 


This  project  demonstrates  that  a  human  factors 
program  in  an  airline  maintenance  environment 
succeeds  only  when  it  adapts  to  the  maintenance 
base’s  specific  environment.  Our  initial  methodology 
of  using  a  workforce/management  team  to  target 
specific  jobs  did  not  produce  successful 
implementations,  despite  its  success  in  many  other 
industries.  Our  airline  partner's  specific  needs 
required  a  different  approach  based  on  involving  the 
maximum  number  of  people,  instead  of  a  small  task 
force,  and  limiting  the  scope  to  one  issue,  i.e., 
communication,  rather  than  searching  broadly  for 
ergonomic  mismatches. 

Focusing  on  communication  brought  potential 
solutions  under  direct  control  of  employees  at  the 
site,  while  still  demonstrating  potential  for  improved 
human  error  rates.  The  use  of  outside  data,  in  this 
case  the  ASRS  reports,  provided  specific  instances  of 
human  factors  needs  which  could  be  related  to  local 
conditions  and  suggested  practical  improvements. 

The  specific  choice  of  the  shift  turnover  log  showed 
how  involvement  of  both  human  factors  professionals 
and  the  inspection  workforce  can  produce  a  practical 
refined  job  aid.  The  new  log  meets  more 
communication  needs  than  its  predecessor  and  has 
good  acceptance  in  the  user  community. 


Establish  mission  and  structure 
Form  human  factors  task  force 
Train  task  force 
Analyze  jobs 
Design  solutions 
Reanalyze  changes 
Transfer  technology. 
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Appendix  6-A 

Ergonomic  Audits  of  Inspection  Tasks 

TO:  :John  Lane 

FROM:  :John  W.  Ditty 

Task  Description:  :Keel  Inspection 

Date:  :4/27/94 

Time:  :10:00  a.m. 

Station:  :Atlanta 

Hangar  Bay:  :RED 

Aircraft  No.  :9153 

M/E  No. 

Q/A  No.  : 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  PRE-INSPECTION/  DOCUMENTATION 

A.  Information  Readability 

1 .  Dot  matrix  printers  with  a  5X7  matrix  of  dot  characters  is  minimally  acceptable  for  reading  purposes.  If 
used,  check  for  character  specifications: 

Minimum  Character  Height  =3.1  mm  to  4.2  mm 
Maximum  Character  Height  =  4.5  mm 

Width/Height  ratio  =3:4-4:5 

IMP:  Do  not  use  lower  case  letters,  since  features  can  get  easily  confused. 

2.  Standards  not  prescribed.  State  "TIME"  &  "QUALITY"  standards  to  ensure  consistent  print  quality. 

B.  Information  Content 
Text 

3.  Feedforward  information  not  provided  to  the  inspector.  Present  information  on 
a:  previous  faults  detected 

b:  locations  of  prior  faults 

c:  likely  fault-prone  areas  for  the  specific  task  &  current  aircraft  under  inspection. 

C.  Information  Organization 

4.  Incorrect  sequencing  of  tasks  in  the  workcard.  Tasks  need  to  be  sequenced  in  the  natural  order  in  which 
the  task  would  be  carried  out  by  MOST  inspectors. 

5.  Avoid  carryover  of  tasks  across  pages  at  ILLOGICAL  points.  Tasks  should  begin  and  end  on  the  same 
page.  For  longer  tasks,  break  into  several  subtasks  with  multiple  sign-offs.  Each  subtask  should  begin  and 
end  on  the  same  page. 

6.  Excessive  number  of  tasks  per  action  statement.  More  than  3  actions/step  increases  the  probability  of 
action  slips. 
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HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  PRE-INSPECTION/  COMMUNICATION 

1 .  No  ongoing  program  to  maintain  adequacy  of  communication  channels. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  PRE-INSPECTION/  VISUAL 
CHARACTERISTICS 

1 .  Fluorescent  bulbs:  "Fair"  to  "Good"  color  rendition  properties.  Color  rendition  is  the  ability  to  distinguish 
true  colors  correctly.  This  is  especially  useful  in  detecting  corrosion  faults.  For  best  results,  consider 
incandescent  bulbs. 

2.  Flicker  exists.  Consider: 

a.  appropriate  shielding  of  ends  of  fluorescent  lamps 

b.  regular  replacement  of  fluorescent  lamps. 

3.  Lighting  fixtures  dirty.  Keep  lighting  fixtures  free/clean  from  dirt/paint. 

4.  No  "Shades/shields"  on  illumination  source.  This  may  cause  "direct"  or  "disability"  glare. 

5.  Illumination  sources  not  working.  Consider  regular  replacement  of  light  sources. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  PRE-INSPECTION/  ACCESS 

ACCESS-STEP  LADDERS 
ACCESS  -  TALL  STEP  LADDERS 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/DOCUMENTATION- 
PHYSICAL  HANDLING  AND  ENVIRONMENTAL  FACTORS 

1 .  Current  light  conditions  inadequate  for  quick  and  easy  reading  of  workcard. 

2.  The  inspector  does  not  sign-off  workcard  after  each  subtask.  This  may  lead  to  errors  of  omission. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/TASK  LIGHTING 

1.  The  average  task  illumination  is  152.50  fc.  and  the  variance  is  2318.75.  The  recommended  task 
illumination  should  be  200.00  fc.  The  variance  is  exceptionally  high. 

2.  Handlamps  deliver  a  max.  of  85  fc.  of  light.  This  illumination  level  is  inadequate  for  "Fine  Inspection." 
Handlamps  also  lack  aiming  control.  Consider  using  of  Standing  Lamping  (Halogen  500  watts-1200  fc.). 

3.  Consider  headlamp  for  hands-free  illumination:  except  in  explosive  environments,  e.g.,  fuel  tank 
inspection. 

4.  The  portable/personal  lighting  equipment  interferes  with  the  inspection  task. 

5.  The  operator  felt  difficulty  in  handling  with  respect  to  the  size  of  the  lighting  equipment. 

6.  The  operator  felt  difficulty  in  handling  with  respect  to  the  weight  of  the  lighting  equipment. 

7.  The  operator  experienced  glare  from  the  task  surface.  Consider: 

a.  reducing  glossiness  of  material 

b.  screening  of  sunlight  penetrations 

c.  repositioning  the  light  source 

d.  use  diffusing  light  sources,  e.g.,  fluorescent  lamps 
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HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/THERMAL 
CHARACTERISTICS 

1 .  The  current  DBT  is  0.00  deg.  cent.  The  recommended  temperature  is  between  20-26  degrees  centigrade. 

2.  The  current  task  has  been  identified  as  having  HIGH  physical  workload.  The  DBT  is  0.00  cent,  and  the 
do  value  for  clothing  is  0.79  do.  The  recommended  DBT  values  for  HIGH  workload  and  do  values 
between  0.75-1.0  are  14-20  deg.  cent.  Consider  change  in  clothing. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/OPERATOR  PERCEPTION  OP 
THERMAL  ENVIRONMENT. 

1 .  The  operator  found  the  summer  temp,  at  the  workplace  to  be  slightly  warm. 

2.  Operator  wanted  the  summer  temp,  at  the  workplace  to  be  cooler  than  the  current  temp. 

3.  Operator  is  generally  not  satisfied  with  the  temp,  at  workplace  during  summer. 

4.  The  operator  found  the  winter  temp,  at  the  workplace  to  be  slightly  cool. 

5.  Operator  wanted  the  winter  temp,  at  the  workplace  to  be  warmer  than  the  current  temp. 

6.  Operator  is  generally  not  satisfied  with  the  temp,  at  workplace  during  winter. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/AUDITORY 
CHARACTERISTICS 

1.  The  maximum  sound  level  at  this  task  is  105  dbA.  Noise  levels  above  90  dbA  indicate  the  need  for 
management  intervention  and  control. 

2.  This  task  involves  verbal  communication.  The  average  noise  level  is  95.60  dbA.  The  distance  of 
communication  is  4.00  feet.  The  noise  level  for  communication  at  a  distance  of  3.5— 6.0  feet  should  not 
exceed  60  dbA. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/NON-DESTRUCTIVE 
TESTING 

1.  NDT  equipment  was  not  easily  maneuverable  during  inspection. 

Displays,  Controls,  and  Knobs 

2.  The  inspector  experiences  division  of  attention.  Consider  using  two  inspectors  for  the  NDT  inspection. 

3.  Visual  checks  are  not  highlighted  by  aural  signals.  Auditory  signals  help  by  providing  redundancy  gain. 
HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/ACCESS-ACTIVITY 

1.  Inspection  affected  by  parallel  work,  e.g.,  opening  or  closing  of  panels,  cleaning  other  inspections,  or 
repair.  Also  check  for  obstruction  due  to  equipment,  e.g.,  tool  boxes,  lighting  equipment,  access 
equipment,  etc. 

2.  The  operator  felt  that  access  was  difficult. 
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3.  The  operator  felt  that  access  was  dangerous. 

4.  Access  equipment  was  repositioned  too  frequently.  This  consumes  a  lot  of  operator  effort.  Consider  using 
multiple  access  equipment. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  INSPECTION/POSTURE 

1 .  The  operator  felt  that  the  workspace  was  constrained. 

The  following  extreme  postures  were  observed  during  the  current  inspection  task;  Urgent  intervention  is 
requested. 

2.  Arms  in  air,  back  bent,  and  loading  on  leg(s). 

3.  Arms  in  air,  back  bent  and  kneeling,  or  laying  or  crawling. 

4.  Arms  in  air,  back  twisted,  and  loading  on  leg(s). 

5.  Arms  in  air,  baek  twisted,  and  kneeling  or  laying  or  crawling. 

6.  Back  bent  and  twisted  and  loading  on  leg(s). 

7.  Baek  bent  and  twisted  and  kneeling,  laying,  or  crawling. 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  PRE-INSPECTION/  SAFETY 

1.  The  inspection  area  is  not  adequately  eleaned  for  inspection.  Consider  appraisal  of  pre-inspection 
processes  like  "open-up"  and  "cleaning". 

HUMAN  FACTORS  MISMATCHES/RECOMMENDATIONS  IN  POST-INSPECTION/FEEDBACK 

1.  Consider  inclusion  of  standard  information  like  ATA  eodes,  station  #,  Sup.  #,  employee  #,  etc.,  in  the 
wbrkcard.  This  considerably  reduees  the  cognitive  load  on  the  inspector. 
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Appendix  6-B 
Ergonomic  Risk  Factors 


1)  Workcards 

•  Card  contrast  changes 

•  Ribbon  changing-establish  preventive  maintenance  program 

•  Graphics-confusion  using  graphics/time  to  get  graphics 

•  Graphics  on  cards-could  one  get  too  reliant  on  cards  and  not  use  the  manual? 

•  Card  content  inaccurate? 

•  Graphics  attached  to  card  until  buy-off 

•  Breaks  between  cards  is  not  good 

•  Use  of  if/then  statements 


2)  Lighting 


9  Fixtures  are  dirty 

•  Need  a  preventive  maintenance  program  for  lighting 

•  Lighting  at  the  back  of  the  hangar  is  inadequate 

9  Color  of  hangar  bays-to  ensure  good  reflectance,  need  a  light  color  floor 

•  Repairs  must  be  performed  by  facilities  department 

3)  Keel  Inspection 

•  Check  task  lighting-cannot  read  workcard 

•  Fuselage  stand  lighting 

•  Handling  lighting  equipment  cords  and  small  lights 

•  Temperature  in  the  summer  is  too  hot 

9  Task  performed  in  very  noisy  environment 

•  Sheet  metal  work  often  interferes  with  task  access 

•  Task  performed  in  a  restricted  space 

•  Difficult  to  get  back  on  to  the  ladder 

9  Task  requires  less-than-optimal  posture 

9  Task  must  often  be  recleaned-cleaners  do  not  understand  necessary  level  of  cleanliness  required  for 
this  task 

•  Cleaners’  work  of  is  not  inspected  before  task  begins 

•  Time  pressure 

4)  PS4  Drain  Box  Inspection 

•  NDT  equipment  design-probe  is  difficult  to  place/equipment  is  not  easy  to  maneuver 

•  Scaffolds/ladders  can  be  slippery/task  is  difficult  to  access 

•  Sign-offs/buy-backs  on  shift  change 
9  Task  light  cords  in  the  way 

•  Check  lighting  levels  on  task 

9  Task  too  hot  when  the  engine  is  still  warm 

9  Cleaning  is  often  inadequate-not  enough  time  to  clean  on  an  overnight  inspection 


86 


Chapter  6 


Appendix  -  Human  Factors  Program 


5)  E&E  Compartment  Inspection 

•  Check  task  lighting 

•  Cannot  read  workcard 

•  Need  fixed  task  lighting  for  a  number  of  tasks-need  to  design  an  appropriate  lighting  fixture 

•  Temperature  high,  due  to  equipment,  in  the  summer 

•  Task  requires  less-than-optimal  postures 

6)  Forward  Accessory  Compartment  Inspection 

•  Task  requires  a  high  ladder-often  difficult  to  find  appropriate  ladder 

•  Requires  a  different  type  of  ladder  than  those  available 

•  Check  task  lighting-use  of  headlamps 

•  Task  is  performed  in  a  restricted  space-difficult  to  access 

•  Task  requires  less-than-optimal  postures 
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Appendix  6-C 
General  Communication 
User  Needs  Analysis 

Your  help  is  needed  to  assess  the  quality  of  internal  and  external  communications  in  the  Hangar  Inspection 
Department.  Here  is  an  excellent  opportunity  for  you  to  help  us  make  improvements  in  the  Inspection  Department 
Communications  System  which  will  give  you  clear  information  on  your  work  assignments  and  make  the 
workplace  less  stressful. 

Please  complete  the  questionnaire  below  and  return  to  the  Atlanta  Safety  Department  by  October  20,  1994. 
Remember,  if  you  do  not  complete  and  return  a  questionnaire,  you  miss  an  opportunity  to  make  a  difference. 

1 .  How  many  years  experience  do  you  have  as  an  inspector? 

2.  Where  (or  from  whom)  do  you  get  necessary  information? 

3.  Is  information  given  to  you  verbally  or  in  written  form? 

4.  Whom  do  you  regularly  pass  information  to? 

5.  How  do  you  pass  information  (verbally  or  in  written  form)? 

6.  Do  you  regularly  have  all  necessary  information  when  working  on  a  task,  or  are  you  constantly  going 
back  for  more  information? 

7.  Do  you  ever  read  the  shift-turnover  log?  If  so,  how  often  do  you  do  so? 

8.  Do  you  ever  write  information  in  the  shift-turnover  log?  If  so,  how  often,  and  under  what  circumstances? 

9.  What  do  you  see  as  the  purpose  of  the  turnover  log? 

10.  If  you  could  design  a  shift- turnover  log,  what  type  of  information  would  you  include? 

11.  Should  the  turnover  log  be  a  SEPTRE  program  similar  to  Hangar  Daily  Stat,  or  book,  or  both? 

12.  Do  you  attend  regular  crew  meetings?  If  so,  who  is  in  attendance  at  these  meetings? 

13.  Do  you  feel  that  regular  crew  meetings  are  informative  and  beneficial,  or  are  they  a  waste  of  your  time? 

14.  Have  you  ever  had  a  problem  caused  by  miscommunication,  either  between  you  and  another  inspector, 
you  and  the  lead  inspector,  you  and  a  manager,  between  you  and  mechanics,  or  you  and  engineering  in 
the  work  area?  If  so,  please  describe. 

15.  How  much  turnover  time  do  you  have  between  shifts?  Is  it  sufficient?  If  not,  how  much  time  is  needed? 

If  additional  space  is  needed,  please  write  your  response  on  the  back  of  the  page,  referencing  the  question  number. 

Thank  you  for  your  time  and  input. 

John  Lane 
Safety  Manager 
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Appendix  6-D 

Speciflcations  for  Proposed  Shift  Turnover  Log 

A)  General  Shift  Turnover  Log 

1)  The  first  section  of  this  log  records  general  shift  information: 

Date:  Enter  the  date  on  which  the  shift  begins. 

To  Be  Read  By:  Circle  the  shift  for  which  this  page  has  been  written:  morning  (1st  shift),  afternoon  (2nd  shift),  or 
night  (3rd  shift).  Each  lead  inspector  should  complete  this  log  for  the  following  shift. 

Lead  Inspector:  Enter  the  name  of  the  acting  lead  inspector  on  the  shift  for  which  this  page  is  intended. 

Manager:  Enter  the  name  of  the  inspection  manager  on  duty  during  the  shift. 

Filled  In  By:  Enter  the  name  of  the  lead  inspector  who  completed  this  page  and  circle  his  or  her  shift. 

Example:  The  day  shift  lead  inspector  should  begin  this  log  for  the  afternoon  shift.  In  the  first  section  of  the  log, 
the  "to  be  read  by"  shift  is  the  afternoon  shift.  The  lead  inspector  is  the  afternoon  lead  inspector's  name.  The 
manager  is  the  afternoon  manager’s  name.  The  day  shift  lead  should  enter  his  or  her  name  and  circle  "morning 
shift"  in  the  "filled  in  by"  box. 

2)  nte  second  section  of  this  log  records  personnel  information.  Information  should  be  recorded  as  it  is  received. 
The  lead  inspector  should  enter  information  in  the  log  that  is  to  be  read  by  the  shift  this  personnel  information 
affects. 

Call-ins  should  be  entered  on  the  log  for  the  shift  the  inspector  was  supposed  to  work. 

Name:  Enter  the  name  of  the  inspector  who  called  in. 

Reason:  Enter  the  reason  the  inspector  called  in,  e.g.,  sick,  family  emergency,  etc. 

Time:  Enter  the  time  the  call  was  received. 

Overtime  should  be  entered  on  the  log  for  the  shift  on  which  the  inspector  is  going  to  work  the  overtime  hours. 
Name:  Enter  the  name  of  the  inspector  who  is  working  the  overtime. 

Reason:  Enter  the  reason  the  inspector  is  working  overtime. 

Time:  Enter  the  number  of  overtime  hours  the  inspector  is  expected  to  work. 

Field  Trips  should  be  entered  on  the  log  for  the  shift  on  which  the  field  trip  begins. 

Name:  Enter  the  name  of  the  inspector  assigned  to  a  field  trip. 

Destination:  Enter  the  destination  of  the  field  trip. 

Departure  Time:  Enter  the  time  the  inspector  departed. 

Return  Time:  Enter  the  time  the  inspector  is  expected  to  return. 
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Example:  If  Inspector  A  is  supposed  to  work  the  midnight  shift  and  calls  in  sick  at  6:00  p.m.,  the  afternoon  shift 
lead  inspector  should  record  this  information  on  the  log  the  night  shift  lead  inspector  is  to  read.  Similarly,  if  day 
shift  Inspector  B  is  asked  to  work  late  (overtime),  this  information  should  be  recorded  on  the  log  the  afternoon 
shift  lead  inspector  is  to  read. 

3)  The  third  section  of  this  log  records  special  instructions  and  general  problems.  This  information,  recorded  by 
the  lead  inspector,  is  to  be  read  by  the  lead  inspector  on  the  following  shift.  Information  intended  for  both 
following  shifts  should  be  recorded  on  both  log  sheets.  The  "resolution,"  "date,"  and  "time"  should  be  completed 
by  the  shift  resolving  the  problem  or  completing  the  project. 

Problem:  Describe  the  problem  or  situation.  Each  problem  on  a  given  day  should  be  numbered 

sequentially. 

Needed  Action 

/Alert:  Enter  the  action  the  oncoming  shift  must  complete  or  describe  the  alert/warning  the  shift  needs 

to  be  aware  of.  Number  the  actions  with  numbers  of  the  problem  to  which  they  refer. 

Resolution:  Describe  the  resolution  determined  or  implemented  for  the  problem  and  include  any  further 

developments  of  a  situation.  Number  the  actions  with  numbers  of  the  problem  to  which  they 
refer. 

Date:  Enter  the  date  the  problem/situation  is  resolved. 

Time:  Enter  the  time  the  problem/situation  is  resolved. 

B)  Aircraft  Log 

1)  The  first  section  of  this  log  records  general  information  about  the  aircraft: 

Aircraft  Number:  Enter  the  number  of  the  aircraft. 

Day:  Enter  the  number  of  days  the  aircraft  has  been  in  the  hangar. 

Shift:  Circle  the  shift  (morning,  afternoon,  night)  completing  this  log. 

Inspectors  Assigned:  Enter  names  of  all  inspectors  assigned  to  this  aircraft  on  this  shift. 

Aircraft  Status:  Circle  the  status  of  this  aircraft;  Line  (not  yet  in  the  hangar).  Initial  Shakedown  (initial 

inspection  in  the  hangar).  Inspection  (performing  scheduled  inspections),  Buy-back  (the 
buy-back  of  non-routine  workcards). 

Enter  any  information  about  this  aircraft  important  for  the  next  shift  to  know  and/or 
understand.  Some  of  this  information  may  also  be  reported  to  the  oncoming  lead 
inspector  and  recorded  in  general  shift  turnover  log. 

2)  The  second  section  of  this  log  describes  ongoing  long-term  projects; 

Project:  Describe  the  project  being  worked  on,  including  the  location  on  the  aircraft,  if  relevant. 

Number  projects  sequentially.  If  more  space  is  needed,  continue  on  the  back  of  the 
page. 


General  Information 
/Notes: 
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Status:  Describe  the  project’s  status,  e.g.,  project  is  30%  complete  or  project  is  waiting  for  a 

specific  part,  etc. 

Needed  Action/Alert:  Describe  any  actions  the  next  shift  must  perform  or  describe  any  warnings/alerts  the 
next  shift  should  be  aware  of  concerning  this  project. 

Inspector:  Enter  the  name  of  the  inspector  who  entered  this  project  into  the  log. 

3)  The  third  section  of  this  log  describes  other  ongoing  projects/problems: 

Inspector:  Enter  the  name  of  the  inspector  who  entered  this  project/problem  into  the  log. 

Project/Problem:  Describe  the  project,  e.g,,  bag-bin  inspection  not  completed,  or  the  problem,  e.g.,  tail 

section  not  clean  enough  to  inspect  at  2:30  p.m.,  that  the  next  shift  must  be  aware  of. 
Number  each  project/problem  consecutively. 

Needed  Action/Alert:  Describe  actions  the  oncoming  shift  should  take  concerning  the  projects  or  problems. 

Resolution:  Describe  the  resolution  to  the  project/problem  that  was  developed  and  implemented. 

Date:  Enter  the  date  the  project  was  completed  or  the  problem  was  resolved. 

Time:  Enter  the  time  the  project  was  completed  or  the  problem  was  resolved. 


91 


Human  Factors  Audit  Program 
FOR  Maintenance _ 

Steven  G.  Chervak  and  Colin  G.  Drury,  Ph.D. 

State  University  of  New  York  at  Buffalo 


This  project's  objective  was  to  provide  a  valid, 
reliable,  and  usable  tool  for  evaluating  human 
factors  in  maintenance  tasks.  The  project  was  part  of  a 
broader  initiative  to  apply  human  factors  to  reduce 
human  error  potential  in  aircraft  inspection  and 
maintenance.  As  Drury  (1994)  pointed  out,  there  is  a 
need  to  move  from  project-level  interventions,  such  as 
better  lighting,  workcards  and  training,  to  higher- level 
process  interventions.  Two  high-level  interventions  in 
this  phase  of  the  FAA/AAM  project  were  (a)  to 
provide  a  tool  for  assessing  the  current  state  of  human 
factors/ergonomics  in  the  hangar  (this  project)  and  (b) 
demonstrating  a  team  approach  to  ergonomic 
interventions. 

The  need  for  an  ergonomics  evaluation  system  has 
been  apparent  for  some  time,  and  manufacturing  audit 
programs  have  been  developed  (e.g.,  Drury,  1990)  to 
provide  a  rapid  overview  of  factors  likely  to  impact 
human/system  mismatches  at  each  workplace.  In  the 
aircraft  inspection  context,  there  is  no  fixed  workplace, 
so  any  audit  program  has  to  start  with  the  workcard, 
rather  than  the  workplace,  as  the  basic  unit.  Such  an 
auditing  system  was  produced  in  conjunction  with  two 
airline  partners  (Lofgren  &  Drury,  1994)  and  tested  for 
both  large  airliners  and  helicopters.  The  system  was 
tested  for  reliability,  and  modified  where  needed, 
before  being  validated  against  human  factors  expert 
judgments.  Significant  agreement  was  found  between 
the  two  cases.  The  system  can  be  used  from  either  a 
paper  data  collection  form  (with  later  data  entry)  or 
directly  from  a  portable  computer.  The  computer  is 
used  to  compare  the  data  collected  against  appropriate 
standards  and  to  print  a  report  suitable  for  use  in  an 
existing  airline  audit  environment.  The  report  allows 
the  airline  to  direct  ergonomic  changes  to  major 
mismatches. 


The  scope  of  this  report  was  to  use  the  Ergonomic 
Audit  for  Aircraft  Visual  Inspection  as  a  starting  point 
for  improvement  and  refinement  to  produce  an 
Ergonomic  Audit  for  Aircraft  Maintenance  (EAAM). 
This  report  details  the  differences  and  similarities 
between  the  two  programs  and  the  process  used  to 
develop  the  new  program/user  interface.  The  EAAM 
was  designed  to  give  an  overall,  generalized 
assessment  of  ergonomic  factors  applicable  to 
performing  a  maintenance  task.  Program  input  and 
output  were  formatted  in  a  way  a  person  unfamiliar 
with  details  of  the  science  of  ergonomics  could 
understand.  This  meant  the  program  had  to  be  easy  to 
use,  had  to  help  guide  the  person  doing  the  audit 
through  the  steps  with  relative  ease,  had  to  describe 
less-familiar  ergonomic  principles,  and  had  to  allow 
the  user  to  access  on-line  help  when  questions  arose. 
The  results  had  to  be  printed  in  an  easily  usable  form 
appropriate  to  the  organization's  needs  and  free  from 
unnecessary  technical  terminology.  As  with  the 
inspection  ergonomics  audit,  the  project’s  overall  aim 
was  to  discover  human/system  mismatches,  not  to 
provide  prescriptive  solutions  to  problems.  Prescriptive 
solutions  still  require  the  depth  of  ergonomic 
knowledge,  which  is  best  provided  by  a  trained 
ergonomist. 

A  task  description  of  a  generic  maintenance  task  must 
be  developed  and  compared  to  that  of  an  inspection 
task  in  order  to  determine  both  differences  and 
similarities  between  the  two.  Once  these  differences 
and  similarities  have  been  identified,  the  inspection 
audit  can  be  modified  to  accommodate  differences  and 
to  provide  an  accurate  tool  with  which  to  begin  the 
ergonomic  audit  and,  eventually,  the  correction 
process. 

From  detailed  task  descriptions  and  task  analyses  of 
inspection  activities,  Drury,  Prabhu  and  Gramopadhye 
(1990)  developed  a  generic  function  description  of 
inspection  (Table  7.1,  next  page).  These  descriptions 
have  been  used  throughout  the  FAA/AAM  project  to 
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structure  inspection  interventions  (Drury,  1994).  Now 
that  these  descriptions  are  to  be  extended  to 
maintenance  tasks,  a  series  of  tasks  were  observed  at 
the  airline  partner's  maintenance  facility.  From  these 
observations,  we  developed  the  equivalent  set  of 
generic  functions  for  maintenance  shown  in  Table  7.2. 

Tables  7. 1  and  7.2  clearly  show  the  many  areas  of 
overlap  between  the  two  activities.  Initiate  (workcards, 
preparation),  parts  of  Access  (getting  to  the  worksite 
with  appropriate  equipment),  Buy-Back  and  Respond 
(final  paperwork)  have  close  parallels  in  these 
activities.  Other  major  functions  are  different,  but  have 
the  same  ergonomic  concerns.  For  example,  the  Search 
function  of  inspection  depends  on  good  lighting  (at 
least  for  visual  inspection)  as  do  the  Diagnosis  and 
Replace/Repair  functions  of  maintenance.  Still,  other 
functions  are  different  between  inspection  and 
maintenance.  For  example,  Opening/Closing  access 
can  require  hand  or  power  tools,  while  Replace/Repair 
can  involve  high  levels  of  force  exertion  or  manual 
lifting:  none  of  these  are  typically  part  of  inspection. 


Fymetion 

visual  Example 

Initiate 

Read  and  understand  workcard. 

Select  equipment. 

Calibrate  equipment. 

Access 

Locate  area  on  aircraft. 

Move  to  worksite. 

Position  self  and  equipment. 

Search 

Move  eyes  (or  probe)  across  area  to 
be  searched.  Stop  if  any  indication. 

Decision 

Re-examine  area  of  indication. 

Evaluate  indication  against  standards. 
Decide  whether  indication  is  defect. 

Respond 

Mark  defect  indication. 

Write  up  non-routine  repair  (NRR). 
Return  to  search. 

Buy-Back 

Examine  repair  against  standards. 

Sign  off  if  repair  meets  standards. 

\  ■  Tasks  1 

Initiate 

Read  and  understand  workcard. 

Prepare  tools,  equipment. 

Collect  parts,  supplies. 

Inspect  parts,  supplies. 

Site  Access 

Move  to  worksite  with  tools,  equipment,  parts,  supplies. 

Part  Access 

Remove  items  to  access  parts. 

Inspect/store  removed  items. 

Diagnosis 

Follow  diagnostic  procedures. 

Determine  parts  to  replace/repair. 

Collect  and  inspect  more  parts  and  supplies. 

Replace/Repair 

Remove  parts  to  be  replaced/repaired. 

Repair  parts,  if  needed. 

Replace  parts. 

Reset  Systems 

Add  fluids  supplies. 

Adjust  systems  to  specification. 

Inspect  adjustments. 

Buy-back,  if  needed. 

Close  Access 

Refit  items  removed  for  access. 

Adjust  items  refitted. 

Remove  tools,  equipment,  parts,  unused  supplies. 

Respond 

Document  repair. 
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The  implication  of  these  differences  was  that  the  audit 
system  for  aircraft  inspection  had  to  be  changed, 
primarily  by  adding  modules  to  cover  maintenance 
tasks.  While  this  change  was  being  introduced,  the 
opportunity  was  taken  to  reconfigure  the  user  interface 
of  the  whole  data  collection  and  analysis  program, 
using  a  more  modem  Windows-based  programming 
language. 


An  audit  program  consists  of  data  collection,  data 
analysis,  and  results  presentation.  Data  collection 
involves  a  series  of  structured  job  observations  and 
recording  these  observations.  Data  analysis  has  a  data 
input  step,  and  a  step  where  data  are  compared  with 
human  factors  standards  and  good  practice.  Finally, 
results  presentation  takes  conclusions  drawn  from  the 
data  analysis  and  provides  them  to  the  user  in  a  useful 
format.  Each  step  can  be  either  a  pencil-and-paper 
activity  or  a  computer-based  activity.  The  audit 
program  previously  developed  for  aircraft  inspection 
and  the  one  developed  here  for  maintenance  tasks  have 
only  specified  computer- allocation  for  the  analysis  and 
results  presentation  steps.  Data  collection  can  either 
use  hard-copy  forms  or  a  portable  computer, 
whichever  best  fits  with  the  organization's  needs.  In 
practice,  many  organizations  prefer  to  use  a  form  for 
initial  data  collection  so  as  to  have  a  permanent  record 
in  a  highly  reliable  medium.  Data  entry  then  consists  of 
transferring  data  from  the  paper  form  to  its  mimic  on 
the  computer's  data  input  module. 

The  audit  program  for  maintenance  inspection  was 
developed  for  an  IBM  personal  computer  as  an 
integrated  program  called  EAAM.  As  with  the 
inspection  audit  program  (ERGO),  a  number  of 
features  were  required  to  ensure  that  the  system  gave 
maximum  benefit  to  the  user  population,  typically, 
maintenance  supervisors  or  quality  auditors.  Any  audit 
program  (Koli  &  Drury,  1995)  must: 

•  be  modular,  so  as  to  include  maximum  coverage 
without  unnecessary  length;  inserting  new 
modules  to  modify  the  checklist  and  program  for  a 
particular  industry  is  straightforward 

•  be  self-explanatory,  so  as  to  minimize  training 
time  for  auditors 

o  be  based  on  standards  from  ergonomics/human 
factors 


•  have  standards  built  into  the  analysis  program, 
rather  than  into  the  checklist,  to  reduce  any 
tendency  to  "bend"  data  in  borderline  cases 

•  rely  on  measurements  and  easily  observable 
conditions  to  reduce  judgment  errors 

•  be  usable  in  different  aviation  environments,  e.g., 
large  fixed-wing  aircraft,  general  aviation  aircraft, 
or  rotary  wing  aircraft,  and  in  different 
maintenance  situations,  whether  line  maintenance 
or  hangar  maintenance. 

In  addition,  a  structure  was  required  to  group  audit 
modules  by  the  human  factors  principle  involved, 
rather  than  by  generic  function.  The  functions  listed  in 
Table  7.2  (previous  page)  ensure  that  coverage  is 
achieved,  i.e.,  all  issues  which  should  be  raised  are 
indeed  part  of  the  audit  system.  Structure  in  the 
program  should  group  together  the  relevant  issues.  For 
example,  the  visual  environment  is  important  in  a 
number  of  functions  of  Table  7.2  (previous  page),  e.g.. 
Part  Access,  Diagnosis,  Replace/Repair,  Close  Access, 
but  the  issues  are  constant,  i.e.,  the  amount  and  quality 
of  lighting.  However,  the  visual  environment  is  only 
one  type  of  environment;  there  are  thermal  and 
auditory  environments,  as  well.  Thus  modules  are 
grouped  in  a  classification  scheme  using  the  following 
four  major  groupings,  following  Prabhu  and  Drury 
(1992)  and  Latorella  and  Drury  (1992): 

•  Information  Requirements  -  documents, 
communication 

•  Environment  -  visual,  auditory,  thermal 

•  Equipment/Job  Aids  -  design  issues,  availability, 
standards 

•  Physical  Activity/Workspace  -  access,  posture, 
safety. 

This  classification  formed  the  basis  of  the  ERGO 
program  and  was  retained  for  EAAM. 

A  second  classification  scheme  was  used  to  reflect  the 
audit  program's  actual  employment.  Some  factors  do 
not  change  during  the  job  and  can  be  conveniently 
evaluated  before  the  job  begins,  e.g,,  workcards' 
quality.  Other  factors  need  the  job  to  be  in  progress 
before  they  can  be  measured,  e.g.,  forces,  noise  levels, 
or  task  lighting.  The  only  module  which  has  to  wait  for 
job  completion  is  the  evaluation  of  feedback  to  the 
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mechanic.  Thus,  the  audit  is  divided  for  convenience 
into  three  phases: 

Pre-Maintenance 

Maintenance 

Post-Maintenance. 

Table  7.3  shows  how  various  modules  are  classified  by 
ergonomics  grouping  and  phase  of  audit.  Clearly,  there 
are  far  more  physical  activity  modules  in  this  system 
than  were  necessary  in  the  inspection  audit  program. 

7.1.1  The  Audit  Program 

The  audit  program  for  maintenance  (MAINAUD) 
will  produce  a  printed  form  for  data  entry,  referred  to 
as  an  Audit  Checklist  (see  Chapter  7  -  Appendix). 
The  data  entry/data  analysis/results  presentation 
program  (EAAM)  reused  some  of  the  inspection 
audit’s  background  data  and  calculations,  e.g.,  in  the 
environment  modules.  However,  we  took  the 
opportunity  to  reprogram  the  whole  audit  system  in 
Visual  Basic  3.0,  instead  of  Turbo  Pascal  6.0.  Turbo 
Pascal  is  a  structured,  high-level  language  with 
multiple  overlapping  windows,  mouse  support,  a 
multi-file  editor,  and  an  enhanced  debugging  facility. 
Visual  Basic  includes  these  factors  and  has  greater 
mouse  support  abilities,  is  more  user-friendly,  and 
can  more  easily  be  expanded  to  incorporate  the 
changes  that  may  occur  in  the  future.  The  advantage 
of  Visual  Basic  is  that  it  allows  a  programmer  to 
create  a  program  that  a  person  with  very  little 


computer  experience  can  use  with  relative  ease. 

Visual  Basic  also  allows  the  flexibility  of  having  the 
final  program  run  on  a  conventional  computer  with 
keyboard  and  mouse  as  input  or  on  a  pen-based 
computer  system  with  stylus  input.  Visual  Basic 
objects,  once  defined  and  coded,  can  be  reused  in 
other  programs,  saving  coding  effort  and  reducing 
coding  errors.  We  chose  Visual  Basic  because  of  the 
similarity  of  its  user  interface  to  other  Windows- 
based  programs.  It  uses  many  of  the  same  symbols 
for  execution  as  the  popular  Microsoft  programs  such 
as  Word,  Excel,  or  Office.  A  person  familiar  with  any 
of  these  programs  should  have  no  problem 
recognizing  similarities  in  Visual  Basic  and  adapting 
to  the  Maintenance  Audit  program,  EAAM. 

The  Title  Screen  (Figure  7.1 ,  next  page)  has  an 
attached  HELP  system  to  provide  assistance  in  using 
the  program.  At  this  level,  the  HELP  screen  offers  a 
program  overview  and  explanation.  Next,  heading 
information  is  required,  e.g.,  the  name  of  the  job,  the 
date,  the  analyst's  name,  etc.  (Figure  7.2,  page  98). 
The  files  for  input  and  report  document  are  specified 
here. 

The  main  program  screen  lists  the  modules  available 
and  asks  the  analyst  to  choose  those  relevant  to  the 
current  job  audit.  Once  the  analyst  chooses  a  set  of 
modules,  each  module  is  presented  (Figure  7.3,  page 
98),  in  turn,  from  the  Pre-Maintenance  phase  through 
the  Post-Maintenance  Phase.  Each  module  (e.g.. 


. 

Human  Factors 
Grouping 

PHASE  I 

Pr©» 

Elfiainteoamce 

Maimtenarace 

Post- 

Rfiairiiteiiiairice 

Information 

Requirements 

1 .  Documentation 

2.  Communication 

6.  Documentation 

7.  Communication 

23.  Buy-Back 

Environment 

3.  Visual 

Characteristics 

8.  Task  Lighting 

9.  Thermal  Characteristics 

1 0.  Thermal  Perception 

1 1 .  Auditory  Characteristics 

Equipment/ 

Job  Aids 

4.  Equipment  Design 

5.  Access  Equipment 

12.  Equipment  Availability 

1 3.  Access  Availability 

Physical  Activity 
Workspace 

14.  Hand  Tools 

15.  Force  Exertion 

16.  Manual  Materials  Handling 

17.  Vibration 

18.  Repetitive  Motion 

1 9.  Physical  Access 

20.  Posture 

21 .  Safety 

22.  Hazardous  Materials 
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Figure  7.4,  page  99)  requires  a  series  of 
measurements  or  classifications.  A  context-sensitive 
HELP  screen  is  available  for  each  module;  it  gives 
detailed  explanations  of  terms  used  and  of 
measurement  procedures  (Figure  7.5,  page  99).  This 
practice  follows  the  recommendations  of  Patel,  Drury 
and  Lofgren  (1994)  for  workcards  in  that  it  supports 
different  kinds  of  users,  from  novice  to  expert.  Each 
module  also  provides  a  comment  screen  (Figure  7.6, 
page  100)  to  allow  the  analyst  to  record  comments  or 
notes. 


As  each  module  is  run,  its  data  are  stored  in  the  file 
the  user  specified  in  the  heading  information  screen. 
When  all  modules  have  been  run,  the  final  report 
document  is  produced,  with  instructions  on  how  to 
obtain  a  hard  copy  through  Windows  software 
(Figure  7.7,  page  100). 
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Us  there  an  overlap  ot  personnel  to  commumcate  prior  shiftwork? 
B  WORK  IN  PROGRESS 
2.  Is  shift  change  work  documented? 


OYos  ONo  On/A 


2.  Is  shift  change  work  documented?  Oves  ONo  On/A 

3.  If  •’Yes'*  to  #  2,  then  are  the  written  documents  communicating  shm  Q  Yas  O  No  O  N/A 

change  legible? 

4  Are  the  communication  channels  evaluated  for  effectiveness?  OYbs  Ono  O  n/A 

5.  Is  there  an  ongoing  program  to  maintain  adequacy  of  communication  Oves  Ono  On/A 

channels? 

S.  Would  the  mechanic  be  considered  a:  O  Novice  O  Expert  O  N/A 

7  Is  the  lead  man  available  for  questions  by  the  mechanic?  OYos  ONo  O  n/A 

0.1s  the  supervisor  available  for  questions  by  the  mechanic?  OYes  ONo  On/A 

I  i  !  NextMaduls 


SMMliiiiKiiSiiliSiSiSSii 


Celcius  to  Fahrenheit  Conversion 


To  do  conversion  click  on  the  arrow! 


j  ® 

Next  Module 

r 
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IRstyrrs  to  Main  Serseo 


Rasytts 
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You  have  oow  compEeted  all  of  the  modules  you  chose.  It  you  would  like  to  go 
back  aBiid  add  more  modules  you  caii  do  so  now  by  clicking  "Return  to  Main 
Screen”. 


In  order  lo  view  or  print  your  report  click  on  "view  results”.  This  will  run  Notepad 
and  automaticayy  call  up  your  tile.  To  print  just  select  "Print”  Irom  the  File  Menu  o 
Notepad,  y  you  would  like  to  return  to  this  screen  Irom  notepad,  select  "Exit"  from 
the  File  menu  ot  Notepad. 
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This  program  is  designed  to  be  run  on  any  IBM 
Personal  Computer  with  at  least  an  INTEL  386 
processor,  4  MB  of  RAM,  DOS  5.0,  and  WINDOWS 
3.1.  The  program  itself  occupies  2  MB  of  hard  disk 
space  in  its  stand-alone  form.  If  a  user  desires  to 
input  data  directly  from  the  job  into  the  program,  a 
portable  computer  is  necessary;  otherwise,  a  desktop 
machine  is  fine.  The  program  can  also  be  run  on  pen- 
based  computers  with  WINDOWS  compatibility. 
[Incidentally,  the  inspection  audit  ERGO  can  also  run 
on  pen-based  systems.] 

The  modules  available  in  EAAM  are  as  follows: 
Pre-Maintenance  Phase 

MODULE  1- 

DOCUMENTATION 
Information  Readability;  Information 
Content,  i.e.,  Text  &  Graphics,  and 
Information  Organization. 

MODULE  2- 

COMMUNICATION 

Between-shift  communication,  availability 
of  lead  mechanics  and  supervisor  for 
mechanics'  questions  and  concerns. 

MODULE  3- 

VISUAL  CHARACTERISTICS 
Overall  lighting  characteristics  of  the 
hanger,  i.e.,  overhead  lighting,  condition  of 
overhead  lighting,  and  glare  from  daylight. 

MODULE  4- 
ELECTRIC/PNEUMATIC 
EQUIPMENT  DESIGN  ISSUES 
Evaluation  of  the  equipment  which  uses 
controls,  i.e.,  ease  of  control,  intuitiveness 
of  controls,  labeling  of  controls  for 
consistency  and  readability. 

MODULE  5- 

ACCESS  EQUIPMENT 
Evaluation  of  ladders  and  scaffold  for 
safety,  availability,  and  reliability. 

Maintenance  Phase 

MODULE  6- 

DOCUMENTATION 

Physical  handling  of  documents  and  the 

environmental  conditions  effecting  the 


documents'  readability,  i.e.,  weather  and 
light. 

MODULE  7- 

COMMUNICATION 

Communication  issues  between  co-workers 
and  supervisors,  and  whether  or  not 
suggestions  are  considered. 

MODULE  8- 
TASK  LIGHTING 

The  overall  lighting  available  to  the 
mechanic  for  completing  the  task.  Evaluates 
points  such  as  light  levels,  whether  personal 
or  portable  lighting  is  used,  and  whether 
lighting  equipment  causes  interference  with 
the  work  task. 

MODULE  9- 

THERMAL  CHARACTERISTICS 
The  current  thermal  conditions  the  task  is 
being  performed  in. 

MODULE  10- 

OPERATOR  PERCEPTION  OF 
THERMAL  ENVIRONMENT 
Operator  perceptions  of  the  work 
environment  at  present,  during  the  summer, 
and  during  the  winter. 

MODULE  11- 

AUDITORY  CHARACTERISTICS 
Determine  if  sound  levels  in  the  current 
work  environment  will  cause  hearing  loss  or 
interfere  with  tasks  or  speech. 

MODULE  12- 

ELECTRICAUPNEUMATIC  EQUIPMENT 
Availability  of  any  electrical/pneumatic 
equipment,  whether  the  equipment  is 
working  or  not,  and  ease  of  using  the 
equipment  in  the  work  environment. 

MODULE  13- 

ACCESS  EQUIPMENT 
Availability  of  ladders  and  scaffolds, 
whether  the  equipment  is  working  or  not, 
and  ease  of  using  the  equipment  in  the  work 
environment. 

MODULE  14- 

HAND  TOOLS 

Evaluates  the  use  of  hand  tools,  whether 
hand  tools  designed  properly  to  prevent 
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disposal  guidelines,  if  the  company  is 
following  current  EPA  requirements  for 
hazardous  material  safety  equipment. 

Post-Maintenance  Phase 


fatigue  and  injury,  and  usability  by  both 
left-  and  right-handed  people. 

MODULE  15- 

FORCE  EXERTION 
Forces  exerted  by  the  mechanic  while 
completing  a  maintenance  task.  Posture, 
hand  positioning,  and  time  duration  are  all 
accounted  for. 


MODULE  16- 

MANUAL  MATERIAL  HANDLING 
Uses  NIOSH  1991  equation  to  determine  if 
the  mechanic  is  handling  loads  over  the 
recommended  lifting  weight. 

MODULE  17- 
VIBRATION 

Amount  of  vibration  a  mechanic  encounters 
for  the  duration  of  the  task.  Determines  if 
there  are  possible  detrimental  effects  to  the 
mechanic  because  of  the  exposure. 

MODULE  18- 

REPETITIVE  MOTION 

The  number  and  frequency  limb  angles 

deviating  from  neutral  while  performing  the 

task.  Takes  into  consideration  arm,  wrist, 

shoulder,  neck,  and  back  positioning. 

MODULE  19- 
ACCESS 

Access  to  the  work  environment.  Whether  it 
is  difficult  or  dangerous,  if  there  is  conflict 
with  other  work  being  performed  at  the 
same  time. 

MODULE  20- 
POSTURE 

Evaluates  different  whole-body  postures  the 
mechanic  must  assume  in  order  to  perform 
the  given  task. 

MODULE  21- 
SAFETY 

Examines  safety  of  the  work  environment 
and  what  the  mechanic  is  doing  to  make  it 
safer,  e.g.,  personal  protective  devices. 

MODULE  22- 

HAZARDOUS  MATERIAL 

Lists  types  of  chemicals  involved  in  the 

maintenance  process,  whether  they  are  being 

used  properly,  if  workers  are  following 


MODULE  23- 
BUY-BACK 

Usefulness  of  feedback  information  to  the 
mechanic  and  whether  buy-back  is  from  the 
same  individual  who  assigned  the  work. 

7.1.2  Audit  Program  Evaluation 

The  EAAM  program  is  only  part  of  an  audit  system. 
Suitable  jobs  must  still  be  chosen  for  auditing,  using 
some  sampling  plan.  The  output  from  the  audit  must 
be  incorporated  into  a  management  structure  which 
will  use  it  effectively  to  improve  job  design.  None  of 
these  issues  are  essentially  different  from  the 
equivalent  issues  for  inspection,  so  they  will  not  be 
repeated  here.  Koli  and  Drury  (1995)  give  details  of 
these  procedures.  More  detail  and  a  discussion  of 
their  relationship  to  the  broader  field  of  human 
factors  can  be  found  in  Koli  (1994). 

Any  tool  designed  for  human  use  should  be  evaluated 
for  its  fit  to  human  capabilities  and  limitations;  this  is 
a  basic  principle  of  ergonomics.  The  audit  program 
for  maintenance  tasks  is  such  a  tool,  and,  like  its 
predecessor  for  inspection,  had  to  be  evaluated.  Koli 
and  Drury  (1995)  tested  the  inspection  audit  program 
ERGO  for  reliability,  i.e.,  whether  different  analysts 
auditing  the  same  job  obtain  the  same  results.  That 
reliability  study  used  three  jobs,  two  on  a  DC-9 
inspection  and  one  on  a  Sikorski  S-58T  inspection. 
There  were  significant  differences  between  the  two 
auditors  tested.  On  further  analysis,  these  differences 
were  shown  to  be  due  mainly  to  inputs  requiring 
auditor  judgment.  These  inputs  were  modified  to 
reduce  the  need  for  judgment.  The  program  was 
retested  on  another  DC-9  task,  showing  no  significant 
differences  this  time  between  auditors. 

Validity  of  a  tool  measures  whether  the  tool  gives  the 
same  output  as  another  trusted  tool.  Koli  (1994) 
tested  the  validity  of  ERGO  by  comparing  its  outputs 
to  those  of  six  ergonomics  experts  viewing  a  video 
tape  of  a  DC-8  power  plant  inspection.  The  audit 
program  always  found  at  least  as  many  ergonomic 
issues  as  any  expert,  and  no  issues  found  by  the 
experts  were  missed  by  ERGO. 
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The  current  program  was  tested  for  both  reliability 
and  validity  in  the  same  way.  In  addition,  its  interface 
was  tested  for  usability,  using  standard  human  factors 
usability  testing  techniques  (McClelland,  1990). 
Initially,  a  single  user  was  observed  and  questioned 
while  using  the  audit  program,  partly  to  assess  its 
usability  and  partly  to  develop  more  detailed 
measures  of  interaction  between  the  user  and  the 
program.  The  particular  user  was  a  member  of  the 
quality  assurance  department  who  regularly 
performed  safety  audits  and  occasional  ergonomics 
audits.  Following  this  analysis,  a  more  detailed 
observation  protocol  was  developed  for  usability 
testing  on  four  other  members  of  the  user  population. 


Two  analysts  observed  four  different  maintenance 
tasks  on  DC-9  aircraft  at  the  airline  partner's 
maintenance  base.  The  tasks  were  the  following: 


1 .  Replace  overhead  passenger  service  unit 

2.  Close  keel  box 

3.  Close  forward  cargo  compartment  access 

4.  Replace  escape  window 

For  each  task,  analysts  used  the  paper  data  collection 


form  as  a  more  severe  test  of  the  audit.  Direct 
computer  entry  of  data  would  have  given  access  to 
HELP  screens.  However,  since  at  least  some  users 
will  want  to  use  paper  data  entry,  this  form  was  used 
as  a  worst  case.  Each  analyst  recorded  answers  for 
each  question  in  each  module  independently  for  later 
comparison.  The  number  of  questions  differed 
between  the  four  tasks,  as  different  modules  applied 
for  each  task.  Note  that  difference  in  results 
between  the  analysts  was  counted,  whether  it  affected 
the  audit  outcome,  or  not. 

The  total  number  of  differences  between  the  two 
analysts'  data  sheets  were  tallied;  the  results  are 
shown  in  Table  7.4.  Also  shown  in  Table  7.4  is  a  X’ 
test  of  the  hypothesis  that  the  number  of  errors  is 
equal  to  zero.  This  is  a  very  stringent  test:  for  125 
questions  only  four  differences  would  be  needed  to 
conclude  that  the  number  of  errors  was  significantly 
different  from  zero. 

As  with  the  initial  reliability  study  of  the  Inspection 
Audit,  the  audit  for  maintenance  was  not  reliable 
enough,  averaging  85%.  The  Cochran  Q  test,  a  robust 
and  strong  test  of  the  differences  between  auditors 
used  to  evaluate  the  reliability  of  the  Inspection 
Audit,  was  performed  on  each  task  to  determine  the 
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# 

Questions 

Diff. 

Reliab. 

X" 

Prob.  I 

1 .  Replace  overhead 

118 

12 

90% 

12.6 

<.001 

passenger  service  unit. 

2.  Close  keel  box. 

163 

22 

87% 

23.6 

<.001 

3.  Close  forward  cargo 

159 

27 

84% 

24.5 

<.001 

compartment  access. 

4.  Replace  escape  v\/indow. 

134 

24 

83% 

26.4 

<.001 

Task 

#  of  Different 
Outcomes 

Cochran's  Q 

Probability 

1 .  Replace  overhead 

10 

1.60 

>0.25  (ns) 

passenger  service  unit. 

2.  Close  keel  box. 

14 

7.14 

<0.01 

3.  Close  fonward  cargo 

10 

0.40 

>0.25  (ns) 

compartment  access. 

4.  Replace  escape  window. 

12 

0.33 

>0.25  (ns) 
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agreement  between  auditors  in  terms  of  output 
results.  For  example,  if  the  percent  of  time  the 
mechanic  spent  in  a  particular  posture  is  estimated  as 
10%  by  one  analyst  and  20%  by  the  other,  but  both 
results  lead  to  the  same  outcome,  a  difference  was 
not  scored.  Table  7.5  (previous  page)  shows  the 
results  of  this  test. 

The  statistic  values  show  significant  differences 
between  the  two  analysts  for  one  of  the  tasks,  with  a 
magnitude  similar  to  those  reported  for  the  same  test 
of  the  Inspection  Audit.  However,  the  non-significant 
findings  on  three  of  the  four  tasks  showed  that  even 
the  first  version  of  this  maintenance  audit  had  been 
based  on  lessons  learned  in  the  inspection  audit.  Note 
that  the  number  of  outcome  differences  was 
considerably  smaller  than  the  number  of  recording 
differences.  Defined  on  outcomes,  reliability  was  in 
fact  92%. 

These  reliability  results  can  be  analyzed  in  more 


detail  to  determine  the  cause  of  each  difference  and, 
hence,  be  used  directly  to  modify  the  EAAM  audit 
program.  Each  difference  was  classified  as  one  of  the 
following: 

Judgment  Error  (J)-  A  magnitude  had  to 
be  judged  by  the  analyst,  e.g..  Was  handling 
the  workcard  difficult? 

Definition  Error  (D)-  A  lack  of  definition 
of  terms  resulting  in  different  assumptions 
by  different  analysts,  e.g.,  Does  the  working 
day  include  lunch  break  (8  hrs)  or  no  lunch 
break  (7  hours)? 

No  Help  on  Form  (H)-  Errors  where  help  is 
available  on  the  program  but  not  on  the 
form,  e.g..  What  is  ulnar  deviation  of  the 
wrist? 

Non-Observation  (N)-  Where  one  analyst 
observed  an  activity,  but  the  other  did  not, 
e.g.,  Is  shift  change  work  documented? 
Other  Errors  (Q)-  All  other  errors,  e.g., 
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where  one  analyst  states  that  the  hand  tool 
requires  a  power  grip,  while  the  other 
analyst  records  nothing. 

Table  7.6  (previous  page)  shows  the  number  of  each 
type  of  difference  counted  for  each  module  of  the 
audit.  As  can  be  seen,  70%  of  all  differences  were 
either  judgment  or  definition  related.  Changes  to 
improve  the  reliability  of  these  questions  are 
relatively  simple,  either  by  replacing  Judgment  with 
measurement  or  by  adding/refining  definitions.  A 
further  12%  of  the  differences  were  due  to  no  help 
facility  on  the  data  collection  form.  Specific  helpful 
expansions  can  be  provided  on  the  form  to  improve 
reliability  here,  too.  Non-observation  errors  and  other 
errors  perhaps  represent  a  minimum  of  errors  (less 
than  2%  of  responses)  which  are  not  simple  to 
correct. 


Overall  reliability  was  in  the  same  range  as  the  initial 
version  of  the  Inspection  Audit.  Specific  changes 
were  made  to  the  program  and  to  the  data  collection 
form  to  secure  the  improvements  required. 

Version  2.0  of  the  Audit  Program  for  Maintenance 
was  developed  and  retested  on  a  single  Job  with  the 
same  two  analysis.  The  rewording  of  questions 
involved  9  of  the  228  questions  in  EAAM.  The  retest 
was  performed  on  the  task  "Replace  first  class  seats" 
on  a  DC-9.  Results  of  the  X“  test  and  Cochran's  Q  test 
are  shown  in  Tables  7.7  and  7.8,  respectively. 

The  reliability  is  now  much  higher  at  93%  when 
calculated  on  number  of  differences  and  the  same  at 
93%  when  calculated  on  number  of  different 
outcomes.  At  this  point  the  reliability  was  considered 
to  be  established. 
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The  ergonomic  audit  program  was  developed  as  a 
rapid  screening  tool  to  identify  ergonomic 
mismatches  in  aircraft  maintenance  tasks.  The 
majority  of  people  using  this  audit  program  will 
have  little  training  and  expertise  in  ergonomics.  In 
order  to  evaluate  the  effectiveness  of  the  program  in 
finding  ergonomic  mismatches,  we  compared  the 
results  of  the  audit  program  to  those  of  four 
practitioners  in  the  field  of  ergonomics.  The  task 
chosen  was  a  Aileron  Removal  on  the  left  wing  of  a 
DC-9  aircraft.  This  task  was  audited  using  the  EAAM 
program  and  simultaneously  videotaped  for  later 
analysis  by  the  ergonomic  practitioners. 

The  EAAM  program  found  55  ergonomic  issues 
which  needed  to  be  addressed.  The  issues  were 
classified  into  10  different  categories  listed  in  Table 
7.9. 

Method:  A  group  of  four  ergonomic  practitioners, 
all  professors  actively  involved  in  conducting 
ergonomic  assessments,  were  provided  with  the 
necessary  documentation  required  to  complete  an 
aileron  removal.  They  were  each  asked  to  view  the 
video  tape  made  of  the  aileron  removal  and  evaluate 
all  aspects  of  the  task,  operator,  equipment, 
documentation,  and  environment  that  they  would 
address  in  evaluating  the  system  for  possible  human 
factor  mismatch  (Koli,  1994). 


Ergoriomic 
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Hand  Tools 
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Safety 

1 

Tote!  '■ 

55  I 

Results:  The  results  of  the  four  subjects  and  that  of 
the  checklist  are  listed  in  Table  7.10  (next  page). 

Note  that  in  some  cases,  for  example 
’’Communication",  the  practitioners  raised  more 
issues  than  the  checklist.  These  "extras"  were  false 
alarms,  where  the  maintenance  task  met  the  standards 
even  though  the  practitioners  thought  it  did  not. 

To  determine  whether  the  checklist  produced  more  or 
less  overall  ergonomic  issues  than  the  practitioners, 
the  differences  between  the  checklist  and  the  mean 
number  of  issues  found  by  practitioners  were 
analyzed  using  a  t-test.  The  value  of  the  t-statistic 
was  t  =  4.57,  which  was  significant  at  p  <  0.01.  This 
indicates  that  there  is  considerable  difference 
between  the  evaluation  of  the  checklist  and  that  of 
the  practitioners,  and  that  the  checklist  found  more 
issues. 

The  relatively  poor  performance  of  the  practitioners 
when  compared  to  that  of  the  checklist  arises  from 
various  sources.  First,  there  is  a  trade-off  between 
direct  observation  and  videotape.  Doing  analyses  by 
direct  observation  allows  the  analyst  to  move  around 
for  the  best  view  and  to  use  three  dimensional  cues. 
This  inflexibility  of  movement  and  unconscious 
editing  by  the  cameraman  performing  the  video 
taping  could  have  resulted  in  loss  of  certain 
information.  One  advantage  of  videotape  analysis  is 
the  analyst  can  play  a  segment  over  or  freeze  action 
in  order  to  analyze  a  situation  more  closely,  but  only 
one  practitioner  used  this  facility.  A  second  reason 
why  the  checklist  outperformed  the  practitioners  is 
because  it  had  been  evolved  by  studying  the  task 
domain  over  an  extended  period  of  time.  All  aspects 
of  the  maintenance  task  were  thoroughly  investigated 
before  the  development  of  the  exhaustive  checklist. 

In  other  words,  the  checklist  was  developed 
specifically  for  aircraft  maintenance  tasks.  The 
practitioners,  on  the  other  hand,  had  to  rely  on 
memory  to  identify  the  issues. 

Overall,  the  checklist  fared  as  well  as,  indeed  better 
than,  ergonomic  practitioners  at  identifying 
ergonomic  mismatches.  However,  one  issue 
involving  safety  was  brought  up  by  practitioners 
which  was  not  identified  directly  by  the  EAAM 
audit:  Safety  aspects  of  the  mechanics  movements. 
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Several  of  the  auditors  made  reference  to  one  of  the 
mechanics’  “jumping”  back  and  forth  between  two 
ladders  in  order  to  complete  the  aileron  removal.  The 
ergonomic  audit  program  does  not  directly  address 
the  issues  of  safety  in  personnel  movement,  but  does 
however  ask  general  safety  questions  of  maintenance 
personnel.  For  example,  “Do  you  feel  access  to  the 
work  area  is  dangerous?”  or  “  Do  you  feel  access  to 
the  work  area  is  difficult?”.  This  audit  was  designed 
so  that  such  general  questions  would  raise  awareness 
of  a  broader  degree  of  personal  safety  issues,  which 
could  then  be  further  investigated  by  ergonomic 
practitioners. 


On  the  basis  of  the  high  reliability  and  validity 
demonstrated  by  the  Maintenance  Audit 
system,  no  further  modifications  were  made  in 
structure  or  content.  Some  interface  changes  have 
been  made  by  Galaxy  Scientific  Personnel,  but  these 
changes  do  not  affect  reliability  or  validity.  For 
1995/96,  it  is  expected  that  the  Inspection  Audit 
(ERGO)  and  the  Maintenance  Audit  (EAAM)  will  be 
combined  with  earlier  audits  into  a  single  audit 
program. 
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MAINTENANCE  PREPARATION 

A.  Information  Requirements 

MODULE  1.  DOCUMENTATION  (Work  Cards) 

a.  Information  Readability 

1 .  Is  the  text  layout  of  this  workcard  consistent  with  the  other  workcards?  (Y/N)__ 

2.  Is  the  text  material  justified  to  the  left  margin?  (Y/N)__ 

3.  Are  typographic  cues  used  for  segregating  important  text  material  in  the  workcard?  (Y/N)_ 

4.  Has  a  simple  block  font  style  been  used  to  print  this  workcard?  (Y/N)_ 

5.  Are  dot-matrix  printers  used  for  printing  workcard?  (Y/N)_ 

6.  If  yes,  its  resolution  matrix  is:  a.  5  X  5 

b. 5X7 

c.  7  X  9  or  higher 

(a/b/c) _ 

7.  Are  the  graphics/attachments  legible  with  reference  to  print  quality?  (Y/N) . 

8.  Are  there  time  &  quality  standards  for  changing  printer  ribbons  S:  toner  cartridges!  (Y/N)  _ 

9  If  yes,  are  the  standards  obeyed!  (Y/N) . 

10.  Have  acronyms/abbreviations  been  used  in  the  workcard?  (Y/N) . 

1 1.  If  yes,  how  many  for  the  entire  task!  a.  less  than  five? 

b.  greater  than  five?  (a/b)  . 

b.  Graphics 

12.  Is  spatial  information  of  body  station  positions  presented  in  pictorial  form?  (Y/N) 

13.  How  arc  figures  represented!  a.  Perspective(3-Dimensional) 

b.  mode  in  which  the  user  sees  it  (a/b) 

14.  Do  figures  have  back  references  to  workcard?  (Y/N) 

15.  Are  figures/graphics  for  mirror-image  tasks  separately  drawn?  (Y/N) 

16.  In  figures/graphics,  are  close-up  views  distinguished  from  distant  views!  (Y/N) 
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c.  Information  Organization 

17.  Is  there  a  definite  ordering/sequencing  of  tasks?  (Y/N) . 

1 8.  Does  task  information  carry-over  to  the  next  page?  (Y/N) . 

19.  What  is  the  maximum  number  of  tasks  per  action  statementl  a.  2 

b. 3 

c.  more  than  3  (a/b/c). 

MODULE  2.  COMMUNICATION 

a.  Shift  Changes 

1.  Is  there  an  overlap  of  personnel  to  communicate  prior  shift  work?  (Y/N) 

b.  Work  in  Progress 

2.  Is  shift  change  work  documented!  (Y/N) 

3.  If  yes,  are  the  written  documents  communicating  shift  change,  legible!  (Y/N) 

4.  Are  the  communication  channels  evaluated  for  effectiveness!  (Y/N) 

5.  Is  there  an  on-going  program  to  maintain  adequacy  of  communication  channels?  (Y/N) 

6.  Would  the  mechanic  be  considered  A)  Novice  or  B)  Expert  (a/b) 

7.  Is  the  Leadman  available  for  questions  by  the  mechanic?  (Y/N) 

8.  Is  the  Supervisor  available  for  questions  by  the  mechanic?  (Y/N) 
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MODULES.  VISUAL  CHARACTERISTICS 

1.  What  is  the  type  of  light  source  used  for  general  illumination? 

a.  incandescent 

b.  fluorescent 

c.  mercury-vapor 

d.  high  pressure  sodium  vapor 

e.  low  pressure  sodium  vapor  (a/b/c/d/e), 

2.  If  fluorescent  bulbs  are  used,  does  flicker  exist?  ( Y/N)  __ 

3.  If  fluorescent  bulbs  are  used,  are  they  installed  in  pairs!  (Y/N)  _ 

4.  Are  lighting  fixtures  free/clean  from  dirt/paint?  (Y/N)  _ 

5.  Are  illumination  sources  provided  with  shades  or  glare  shields!  (Y/N)  „ 

6.  Are  all  the  illumination  sources  working!  (Y/N)  _ 

7.  Is  there  indirect  glare  from  the  source?  (Y/N)  _ 

8.  Is  the  general  lighting  source  within  the  line  of  sight?  (Y/N)  _ 

MODULE  4.  ELECTRICAUPNEUMATIC  EQUIPMENT  DESIGN  ISSUES 

1.  Are  controls  requiring  precision  performed  manually?  (Y/N)  _ 

2.  Do  selector  switches  have  fixed  scales  and  moving  pointers?  (Y/N)  __ 

3.  Are  toggle  switches  used  in  sequence,  mounted  in  a  horizontal  array?  (Y/N)  _ 

4.  Are  controls  labeled  with  all  "words'"  or  "symbols"!  (Y/N)  _ 

5.  Are  labels  typographically  consistent!  (Y/N)  _ 

6.  Do  push  buttons  prevent  slipping  of  fingers  (eg.,  surface  texture,  shape  of  knob  etc.)?  (Y/N)  __ 

7.  Do  push  buttons  have  an  audible  click  or  snap  feel  to  indicate  control  action?  (Y/N)  _ 

8.  Are  edges  of  knobs,  dials,  switches  or  instrument  rounded?  (Y/N)  _ 

9.  Are  labels  readable  in  all  weather  conditions?  (Y/N)  _ 

10.  Have  abbreviations  been  avoided  on  labels  wherever  possible?  (Y/N)  _ 

11.  Are  emergency  controls  clearly  distinguished  from  normal  controls?  (Y/N)  __ 

12.  If  the  control  function  is  RAISE,  is  the  movement  of  the  control  UP!  (Y/N)  _ 
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13.  If  control  function  is  ON,  is  movement  RIGHT,  CLOCKWISE,  FORWARD  or  PUSH1(Y/N)  _ 

14.  If  control  function  is  INCREASED,  is  movement  RIGHT,  CLOCKWISE  or  FORWARDliyn^ 

15.  If  control  function  is  RIGHT,  is  the  movement  RIGHT  or  CLOCKWISE"}  (Y/N)  — 

16.  If  the  control  is  RETRACT,  is  the  movement  UP,  REARWARD  or  PULLl  (Y/N)  __ 

MODULE  5.  ACCESS  EQUIPMENT  -  LADDERS,  SCAFFOLDS 

1.  Do  ladders/scaffolds  have  non-skid  surfaces  on  landings?  (Y/N)  _ 

2.  Do  ladders/scaffolds  have  safety  screens  behind  open  stairs  and  at  landings?  (Y/N)  __ 

3.  Do  ladders  have  hand  rails"!  (Y/N)  __ 

4.  What  is  the  cross  section  of  the  hand  rails?  a.  circular 

b.  rectangular 

c.  other  (a/b/c)  _ 


5.  What  is  the  angle  of  inclination  of  the  ladder  with  the  horizontal?  A= - ^ 

6.  What  is  the  riser  height?  R  = - inches 

7.  What  is  the  tread  length?  X  = - inches 

8.  If  non-tread  ladders  are  used:  what  is  the  distance  between  vertical  rails"!  Y  = - inches 

9.  If  non-tread  ladders  are  used:  What  is  the  cross  section  of  the  rungs"!  a.  circular 

b.  rectangular 

c.  other  (a/b/c) _ 


10.  If  non-tread  ladders  are  used:  What’s  the  cross 


section  of  the  vertical  rails"!  a.  circular 

b.  rectangular 

c.  other  (a/b/c) 
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ACCESS  EQUIPMENT  -  PORTABLE  LADDERS  (Step  Ladders  &  Tall  Step  Ladders) 


Step  ladders 

1 1 .  What  is  the  height  of  the  step  ladder?  H  = _ inches 

12.  Does  the  step  ladder  have  non-slip  treads!  (Y/N) _ 

13.  Does  the  step  ladder  have  rubber  feet!  (Y/N) _ 

Tall  Step  Ladders 

14.  Does  the  tall  step  ladder  have  braces  on  the  lower  steps?  (Y/N) _ 

15.  Do  the  folding  braces  of  the  ladder  have  locking  detents!  (Y/N) _ 
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A.  Information  Requirements 

MODULE  6.  DOCUMENTATION  (Physical  Handling  &.  Environmental  Factors) 

1.  When  did  the  mechanic  last  perform  this  task?  a.  a  day  ago 

b.  a  week  ago 

c.  a  month  or  more  (a/b/c) 

2.  Does  the  Mechanic  read  the  workcard?  (Y/N) 

3.  Do  you  feel  the  information  content  of  the  workcard  complete  with  respect  to 

the  scope  of  the  taskl  (Y/N) 

4.  Do  you  feel  a  novice  mechanic  can  understand  this  current  workcard?  (Y/N) 

5.  Do  you  feel  there  is  any  handling  difficulty  with  respect  to  the  size  of  the 

workcard/graphic  attachments  while  conducting  maintenance?  (Y/N) 

6.  Do  you  feel  there  is  adequate  readability  in  the  current  light  conditions?  (Y/N) 


7.  Is  maintenance  being  conducted  in  conditions  of:  a.  wind  (Y/N) 

b.  rain  (Y/N) 

c.  snow  (Y/N) 

8.  Does  the  mechanic  sign-off  ihc  workcard  after  each  subtask?  (Y/N) 

9.  Do  writing  tools  facilitate  writing  in  all  positions?  (Y/N) 

MODULE  7.  COMMUNICATION 


(Maintenance  person  to  be  asked  the  following  questions) 

1.  How  easy  is  communication  (work-related)  with  co-worker?  a.  very  easy 

b.  adequate 

c.  very  difficult  (a/b/c) 

2.  Did  you  get  explicit  verbal  instructions  from  the  supervisor?  (Y/N)  _ 

3.  How  easy  is  communication  with  supervisor?  a.  very  easy 

b.  adequate 

c,  very  difficult  (a/b/c) 


4.  Are  you  given  feedback  when  you  are  not  performing  up  to  the  standard?  (Y/N) 

5,  Are  you  encouraged  to  help  identify  error  likely  situations  in:  a.  existing  design  (Y/N) 

b.  maint.  proc.  (Y/N) 


6.  Are  the  suggestions  reviewed? 


(Y/N) 
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MODULE  8.  TASK  UGHTING 

1.  What  type  of  work  is  being  audited?  a.  ordinary  maintenance 

b.  detailed  maintenance 

c.  fine  maintenance  (a/b/c). 


2.  Does  mechanic  look  from  bright  to  dark  places  routinely?  (Y/N) _ 

3.  Indicate  the  light  levels  taken  from  4  zones  during  the  task.  Zone  1  = _ fc 

Zone  2  = _ fc 

Zone  3  = _ fc 

Zone  4  = _ fc 

4.  What  type  of  light  source  is  used  as  portable  lighting  equipment?  a.  hand  lamp  (Y/N) _ 


b.  standing  lamp  (Y/N) _ 

5.  What  type  of  light  source  is  used  as  personal  lighting  equipment?  a.  2D  cell  flashlight 

b.  3D  cell  flashlight 

c.  4D  cell  flashlight 

d.  Headlamp 

e.  Other  (a/b/c/d/e) _ 


6.  Does  the  portable  or  personal  lighting  equipment  interfere  v^ith 

the  maintenance  task?  (Y/N) - 

7.  Do  you  feel  any  difficulty  in  handling  with  respect  to  the  size  of 

the  lighting  equipment?  (Y/N) - 

8.  Do  you  feel  any  difficulty  in  handling  with  respect  to  the  weight 

of  the  lighting  equipment?  (Y/N) - 

9.  Do  you  experience  discomfort  glare  from  the  task  surface  ?  (Y/N) _ 

10.  Do  you  experience  discomfort  glare  from  workcard  surfacel  (Y/N) _ 

11.  Are  there  excessive  contrasts  between  different  colors  in  the  task  area?  (Y/N) _ 
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MODULE  9.  THERMAL  CHARACTERISTICS 

Measurement  tools:  Drv  and  Wet  bulb  thermometer  and  an  anemometer  to  measure  the  wind  speed. 


1 .  Describe  the  physical  workload/muscular  ejforti 

a.  low 

b.  moderate 

c.  high 

(a/b/c) _ 

2.  What  is  the  wind  speed? 

Mph 

3.  The  air  temperature  is  approximately? 

op 

4.  What  is  the  Humidity  of  the  hangar? 

% 

MODULE  10.  OPERATOR  PERCEPTION  OF  THERMAL  ENVIRONMENT 

This  module  evaluates  the  perceptions  of  the  operators  to  climate  changes.  All  the  questions  in  this  module  are  to 
be  addressed  to  the  inspector  performing  the  task. 


1.  How  do  you  feel  nowl 

1  2  3 

1  1  1 

4 

1 

5 

1 

Scale  reading  _ 
6 

1 

7 

_ 1 

hot  warm  slightly  warm 

neutral 

slightly  cool 

cool 

cold 

2.  Indicate  how  you  would  like  to  be  now? 

SUMMER 

a.  warmer 

b.  cooler 

c.  no  change 

(a/b/c) 

3.  How  do  you  feel  during  summer! 

1  2  3 

1  1  1 

4 

1 

5 

1 

Scale  reading  _ 
6 

1 

7 

_ 1 

hot  warm  slightly  warm 

neutral 

slightly  cool 

cool 

cold 

4.  Indicate  how  you  would  like  to  be  during  summer? 

a.  warmer 

b.  cooler 

c.  no  change 

(a/b/c) 

WINTER 

6.  How  do  you  feel  during  winter! 

1  2  3 

1  1  1 

4 

1 

5  6 

1 

Scale  reading  _ 
7 

1 

_ 1 

hot  warm  slightly  warm 

neutral 

slightly  cool 

cool 

cold 

7.  Indicate  how  would  you  like  to  be  during  winter?  a.  warmer 

b.  cooler 

c.  no  change  (a/b/c) 
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MODULE  11.  AUDITORY  CHARACTERISTICS 


Measurement  Tools:  Sound-level  meter  that  measures  sound  in  decibels. 


1.  The  noise  levels  recorded  over  the  entire  inspection  task  duration  are: 


Reading#  1 _ dBA 

Reading#  2  dBA 

Reading#  3 _ ^dBA 

Reading#  4 _ dBA 

Reading#  5 _ dBA 


2.  At  each  reading,  the  main  source  of  noise  from: 

a)  pneumatic  tools 

b)  music 

c)  conversation 

d)  engines 

e)  passing  aircraft 

f)  other 


answer  (a,b,c,d,e,f) 

Reading  #  1 _ 

Reading  #  2 _ 

Reading  #  3 _ 

Reading  #  4 _ 

Reading  #  5 . 


3.  What  is  the  approximate  exposure  time  to  the  existing  noise  levels? 


.  hours/day 


4.  Does  the  maintenance  person  wear  earplugsl  (Y/N) _ 

5.  Does  the  maintenance  person  wear  earmujfsl  (Y/N) _ 

6.  The  maximum  distance  which  the  maintenance  person  needs  to  communicate  verbally  is? _ feet 

7.  Is  there  a  high  pitch  noise  component?  (e.g.,.  over  2000  Hz)  (Y/N) _ 

8.  Is  the  main  source  of  noise  from  other  workstations?  (Y/N) _ 

MODULE  12,  ELECTRICAUPNEUMATIC  EQUIPMENT 


Availability 


1.  Is  equipment  available! 

2.  Is  the  equipment  working  at  all  times? 


3.  If  no,  are  there  any  satisfactory  substitute  arrangements! 


4.  Is  electrical/pneumatic  equipment  easily  maneuverable  during  maintenance? 
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Displays.  Controls,  Knobs 

5.  Can  you  easily  understand  all  the  labels/display  menusl  (Y/N)  _ 

6.  Are  control  elements  easily  differentiated  by  touch?  (Y/N)  _ 

7.  Are  control  movements  as  short  as  possible?  (Y/N)  _ 

8.  Is  there  division  ofattentionl  (Y/N)  _ 

MODULE  13.  ACCESS  EQUIPMENT 

Availability 

1 .  Is  correct  access  equipment  available!  (Y/N)  _ 

2.  If  no,  is  satisfactory  substitute  equipment  available?  (Y/N)  _ 

3.  The  access  equipment  is:  a.  fixed 

b.  movable 

c,  both  of  the  above  a/b/c)  _ 

4.  If  movable,  is  it  easily  maneuverable!  (Y/N) , 

MODULE  14.  HAND  TOOLS 

1.  Is  there  shoulder  adduction  during  tool  operation?  (Y/N) . 

2.  Is  forearm  fully  extended  during  tool  operation?  (Y/N) . 

3.  Does  tool  operation  involve  noticeable:  a)  Wrist  ulnar  deviation?  (Y/N) . 

b)  Wrist  radial  deviation?  (Y/N) . 

c)  Wrist  flexion?  (Y/N) . 

d)  Wrist  extension?  (Y/N) . 

4.  Does  the  tool  vibrate  perceptibly?  (Y/N) 

5.  Can  the  tool  be  used  by  both  left  and  right  handed  people?  (Y/N) 

6.  Does  the  tool  handle  end  in  the  palm?  (Y/N) 

7.  For  power  tool,  does  the  tool  handle  provide  electrical  insulation?  (Y/N) 

8.  Does  the  tool  handle  provide  heat  insulation?  (Y/N) 

9.  Does  the  tool  handle  have  sharp  edges  or  corners?  (Y/N) 

10.  Is  the  tool  handle  compressible?  (Y/N) 

1 1 .  Is  the  tool  handle  hard  enough  to  resist  embedding  of  particles?  (Y/N) 

12.  Is  the  tool  grip  non-absorbent  to  sweat,  oil,  grease,  etc.?  (Y/N) 
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13.  Is  a  heavy  grip  needed  to  avoid  slippage?  (Y/N) . 

14.  Are  there  any  unguarded  pinch  points  on  the  tools?  (Y/N) . 

15.  Are  there  stops  to  prevent  the  handles  from  fully  closing?  (Y/N) . 

16.  The  type  of  activating  trigger  is  :  a.  single  finger?  (Y/N) . 

b.  multiple  finger  strip?  (Y/N) . 

c.  thumb?  (Y/N) . 

17.  If  a  thumb  operated  trigger  is  used,  is  the  thumb  hyperextended?  (Y/N) , 

18.  Is  the  trigger  very  frequently  used?  (Y/N) . 

19.  The  grip  on  the  tool  is:  a.  pulp  pinch 

b.  lateral  pinch 

c.  power  grip  (a/b/c) 

20.  If  the  tool  is  heavy  is  it  supported  or  counter  balanced?  (Y/N) . 
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MODULE  15.  FORCE  EXERTION 

1.  Does  the  task  involve:  Horizontal  pushing? 

Horizontal  pulling? 

Vertical  pushing? 

Vertical  pulling? 

2.  Does  the  task  involve  use  of  One  arm? 

Both  arms? 

3.  Is  the  type  of  grip:  a.  power  grip? 

b.  hook  grip? 

c.  finger  pinch  grip? 

4.  Vertical  level  of  first  force  application  :  a.  Above  head  height 

b.  Head  height 

c.  Shoulder  height 

d.  Elbow  height 

5.  Muscle  groups  involved  in  the  task:  a.  whole  body 

b.  primarily  arm  and  shoulders 

6.  Is  the  person’s  arm  moving  while  the  force  is  being  applied? 

7.  What  is  the  force  being  applied? 


IM 


(Y/N) _ 

(Y/N) _ 

(Y/N) _ 

(Y/N) _ 

(Y/N) _ 

(Y/N) _ 


(a/b/c) 


(a/b/c/d) _ 


(afij) _ 

(Y/N) _ 

_ (Kg.) 
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MODULE  16.  MANUAL  MATERIAL  HANDLING 

1 .  Do  loads  have  proper  handles?  (Y/N) _ 

2.  Can  these  handles  be  used  by  the  whole  hand?  (Y/N) _ 

3.  If  protective  clothing  is  indicated,  is  it  provided?  (Y/N) _ 

4.  Is  the  task  area  clear  of  obstructions?  (Y/N) _ 

5.  Is  the  floor  clean,  dry  and  non-slip?  (Y/N) _ _ 

6.  Is  the  area  for  setting  down  the  load  clear?  (Y/N) _ 

NIOSH  EQUATION 

1  .  What  is  the  objects  weight?  (kg) _ 

2.  Frequency  of  Task?  (Lift/Min) 

3.  Hand  distance  away  from  body  at  start?  (cm) _ _ 

4.  Hand  height  at  start?  (cm) _ 

5.  Hand  distance  away  from  body  at  conclusion?  (cm) _ 

6.  Hand  height  at  conclusion?  (cm) _ 

7.  Width  of  Object?  (cm) _ 

8.  Back  Rotation  angle?  (Deg.) _ 

9.  Task  Duration?  (Hrs.) _ 

5.  Is  the  floor  clean,  dry  and  non-slip?  (Y/N) _ 

6.  Is  the  area  for  setting  down  the  load  clear?  (Y/N) _ 

MODULE  17.  VIBRATION 

1.  Is  hand-arm  vibration  present?  (Y/N) _ 

2.  Are  anti-vibration  tools  being  used?  (Y/N) _ 

3.  Are  anti-vibration  gloves  being  used?  (Y/N) _ 

4.  Are  workbreaks  provided  to  avoid  constant  vibration  exposure?  (Y/N) _ 

5.  Do  hands  remain  warm  while  working?  (Y/N) _ 

6.  Can  the  tool  be  supported  or  rested  while  working?  (Y/N) _ 
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7.  Does  worker  experience:  a.  tingling  of  the  digits  (finger)  ?  (Y/N) _ 

b.  numbness  of  the  digits?  (Y/N) _ 

c.  blanching  of  digits?  (Y/N) _ 

8.  What  is  the  vibration  frequency?  (HZ) - 

9.  What  is  the  duration  of  maximum  continuous  vibration  exposure?  (Min) - 

10.  What  is  the  total  duration  of  vibration  exposure  on  this  shift?  (Min) - 

11.  What  is  the  vibration  acceleration?  (m/s'^2) - 


MODULE  18,  REPETITIVE  MOTION 


1.  Does  the  task  require  the  following  to  be  performed? 

a.  Reach  with  arms  above  shoulder  level  (Y/N) - 

b.  Work  with  arms  above  shoulder  level  (Y/N) - 

c.  Reach  behind  the  body  (Y/N) - 

d.  Inward  rotation  of  forearm  with  bent  wrist  (Y/N) - 

e.  Outward  rotation  of  forearm  with  bent  wrist  (Y/N) - 

f.  Ulnar  deviation  of  wrist  combined  with  supination  (Y/N) - 

g.  Radial  deviation  of  wrist  combined  with  pronation  (Y/N) - 

h.  Flexion  of  wrist  (Y/N) - 

i.  Extension  of  wrist  (Y/N) - 

j.  "Clothes  wringing"  motion  with  hands  (Y/N) - 

k.  Hand/wrist  contacting  sharp  edges  (Y/N) - 

l.  Flexion  of  the  back  (Y/N) - 

m.  Extension  of  the  back  (Y/N) - 

n.  Flexion  of  the  shoulders  (Y/N) - 

o.  Extension  of  shoulders  (Y/N) - 

p.  Flexion  of  neck  (Y/N) - 

q.  Extension  of  neck  (Y/N) - 


2.  If  a  tool  is  being  used: 

a.  Can  the  location  of  the  tool  be  adjusted? 

b.  Is  the  tool  suspended? 

c.  Is  the  tool  handle  made  of  non-metailic  material? 


(Y/N) 

(Y/N) 

(Y/N) 
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MAINTENANCE  PHASE 

MODULE  J9.  ACCESS 

1 .  Is  there  any  conflict  due  to  parallel  workl  (Y/N) 

2.  Do  you  think  access  is:  a.  difficult?  (Y/N) 

b.  dangerous?  (Y/N) 

3.  How  often  was  access  equipment  repositionedl  a.  1  or  2  times  in  the  entire  task 

b.  3  or  more  times  (A/B) 

MODULE  20,  POSTURE 

1 .  Do  you  feel  that  the  workspace  is  constrained!  (Y/N) 

2.  How  often  were  the  following  postures  adopted  by  Mechanic  during  the  task? 


# 

\  body  part  positions 

percentage  of  total  task  time 

UPPER 

LIMBS 

BACK 

LOWER  LIMBS 

0% 

0%  - 
10% 

10% - 
25% 

above 

25%, 

1 

arm(s)  in 
air 

back  bent 

leg(s)  bent 

2 

arm(s)  in 
air 

back  bent 

kneeling/crawling/laying 

3 

arm(s)  in 
air 

back  twisted 

leg(s)  bent 

4 

arm(s)  in 
air 

back  twisted 

kneeling/crawling/laying 

5 

arm(s)  in 
air 

back  bent  and 
twisted 

leg(s)  bent 

6 

arm(s)  in 
air 

back  bent  and 
twisted 

kneel ing/cra  wiing/laying 

0%  -  never  observed  10%  -  25%>  -occasionally  observed 

0%  -  10%o  -  seldomly  observed  above  25%>  -  frequently  observed 


MODULE  2L  SAFETY 

1 .  Is  the  work  area  free  of  clutter,  dirt,  oils,  etc?  (Y/N) 

2.  Are  safety  attachments  used  when  the  mechanic  performs  maintenance  at  heights?  (Y/N) 

3.  Is  the  maintenance  person  wearing  safety  shoes?  (Y/N) 

4.  If  task  requires,  is  the  maintenance  person  wearing  eye  protection?  (Y/N) 
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MODULE  22.  HAZARDOUS  MATERIAL 

1 .  Is  training  provided  for  proper  handling  and  clean  up  of  hazardous  materials?  (Y/N) 

2.  Are  all  hazardous  materials  properly  labeled  with  type  and  caution  information?  (Y/N) 

3.  Are  eyewash  stations  available  for  emergency  use?  (Y/N) 

4.  Are  shower  stations  provided  for  emergency  use?  (Y/N) 

5.  Are  all  hazardous  materials  properly  labeled  with  type  and  cautions?  (Y/N) 

6.  Were  hazardous  material  signed  out  and  weighed?  (Y/N) 

7.  Were  hazardous  material  signed  in  and  weighed?  (Y/N) 

8.  If  unused  material  was  discarded,  was  it  done  properly?  (Y/N) 

9.  Does  Work  Card  give  proper  Hazardous  material  Identification  #?  (Y/N) 

10.  Hazardous  Material  being  used  is  in  the  form  of;  a)  Paint 

b)  Epoxy 


c)  Cleaning  Agent 

d)  Lubricant 

e)  More  than  one 


0  Others  (a/b/c/d/e/f) 

1 1 .  Is  safety  equipment  (corresponding  to  the  type  of  hazardous  material)  being  used?  (Y/N) - 

12.  Is  the  recommended  safety  equipment  readily  available?  (Y/N) - 

13.  Does  the  safety  equipment  cause  restriction  in  movement?  (Y/N) - 


14.  Is  the  General  Maintenance  Manual  available  for  review  of  Hazardous  Material  use?  (Y/N) _ 

15.  What  %  of  total  task  time  are  the  hazardous  materials  being  used?  a)  10%  -  24% 

b)  25%  -  49% 

c)  50%  -  74% 

d)  75%  -  99% 

e)  100%  (a/b/c/d/e) 

16.  Does  the  use  of  a  hazardous  material  intrude  on  other  workers?  (i.e.,  fumes,  aerosol)  (Y/N) - 
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POST  MAINTENANCE 

MODULE  23,  BUY-BACK  FOR  ROUTim  MAINTENANCE 

1.  Was  the  maintenance  task  required  to  be  bought  back  by: 

a)  the  initial  inspector? 

b)  any  Inspector  (besides  initial  inspector)? 

c)  maintenance  foreman? 

d)  maintenance  person  himself?  (a/b/c/d) 

2.  Did  the  task  pass  buy  -  back  on  the  first  try?  (Y/N)  _ 

3.  If  No  to  question  2,  was  the  same  inspector  used  for  the  latter  attempts  at  buy-back?  (Y/N)  _ 

4.  Was  the  maintenance  person  present  when  the  buy  back  was  done?  (Y/N)  _ 

5.  If  "Yes”  to  #4,  was  feedback  information  given  to  the  maintenance  person?  (Y/N)  __ 

6.  If  "No"  to  #4,  was  maintenance  person  informed  of  discrepancies  by  written  notice?  (Y/N)  _ 

7.  Does  the  maintenance  person  feel  feedback  information  is  informative  and  useful?  (Y/N)  _ 

8.  Is  the  supervisor  available  for  questions  by  the  maintenance  person?  (Y/N)  „ 
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Patel,  Prabhu,  and  Drury  (1993)  describe  a  work- 
card  as  ’’the  prime  source  of  on-line  directive  and 
feedforward  information  in  aircraft  inspection.  It  is 
the  primary  document  that  starts  the  inspection  and 
serves  as  a  major  influencing  factor  on  inspection 
performance"(p-l)-  The  workcard  can  also  be  viewed 
as  a  checklist  that  aids  the  mechanic  in  recalling  all 
the  numerous  tasks  to  be  performed  in  a  check.  Once 
a  task  or  group  of  tasks  is  finished,  the  mechanic  or 
inspector  is  required  to  sign  it  as  being  satisfactorily 
completed.  As  the  workers  perform  these  tasks  re¬ 
peatedly,  there  is  a  tendency  to  perform  them  at  least 
partially  from  memory,  with  a  block  of  sign-offs 
made  at  a  convenient  time.  This  is  not  how  workcards 
are  intended  to  be  used,  and  such  use  can  result  in  er¬ 
rors.  Since  the  safety  of  civil  aircraft  is  highly  de¬ 
pendent  on  reliable  inspection,  we  undertook  an 
analysis  of  how  workcards  are  presently  used  and 
how  workcards  design  affects  their  use  and  the  subse¬ 
quent  potential  for  error. 

8.1.1  Checklist  Objectives 

Workcards  and  other  forms  of  checklists  are  common 
throughout  the  aviation  industry.  In  addition  to  work- 
cards  being  used  for  all  inspection  and  maintenance 
tasks,  flight  crews  use  checklists  to  prepare  the  air¬ 
craft  for  each  new  stage  of  a  flight.  Degani  and  Wie¬ 
ner  (1990;  1993)  reviewed  the  role  of  checklists  in 
the  cockpit,  the  potential  effects  of  their  design,  and 
sociotechnical  factors  affecting  their  use.  Although 
the  content  of  flight  deck  checklists  differs  substan¬ 
tially  from  those  for  maintenance  and  inspection,  the 
checklists’  objectives  (as  Degani  and  Wiener  describe 
them),  as  well  as  many  of  their  design  concepts  and 
performance  factors,  are  similar. 

Degani  and  Wiener  defined  checklist  objectives  that 
are  pertinent  to  aircraft  maintenance:  to  assist  the  user 
in  recalling  procedures,  to  outline  a  convenient  se¬ 
quence  for  motor  movements  and  eye  fixations,  to  al¬ 


low  mutual  supervision  within  crews,  to  distribute  tasks 
among  crew  members,  and  to  act  as  a  quality  control 
tool  for  management  and  government  regulators 
(Degani  and  Wiener,  1990,  p.7).  The  first  objective  of 
a  workcard  is  to  remind  mechanics  or  inspectors  of 
items  to  be  checked;  any  type  of  job  aid  shares  this 
goal.  By  providing  information  externally,  a  job  aid  re¬ 
duces  the  information  a  person  must  store  and  process 
(Swezey,  1987).  Listing  tasks  in  an  order  providing  a 
convenient  sequence  of  motor  movements  should  re¬ 
duce  the  time  spent  accessing  the  task  areas.  Work- 
cards  also  provide  written  records  of  tasks  to  be 
performed  and  ease  the  supervision  and  distribution  of 
tasks.  Finally,  sign-offs  of  tasks  on  a  workcard  verify 
that  the  work  is  complete,  as  dictated  by  the  airline  and 
by  FAA  regulations.  Workcards  used  in  aircraft  main¬ 
tenance  and  inspection  tasks  should  meet  these  check¬ 
list  objectives.  For  this  project,  we  analyzed  methods 
maintenance  technicians  use  to  perform  different  levels 
of  checks  to  determine  if  their  workcards  met  these 
goals.  More-detailed  B-,  C-  and  D-checks  have  fewer, 
larger  tasks  on  each  workcard.  Lower-level  checks  (A- 
checks  and  below)  were  the  main  focus  of  this  study 
because  they  typically  consist  of  larger  lists  (20-100 
items)  of  relatively  short  tasks.  These  are  what  is  called 
"checklists."  Although  people  performing  these  checks 
are  classified  as  mechanics,  these  tasks’  functions  are 
associated  with  inspection,  i.e.,  checking  whether  spe¬ 
cific  aircraft  features  meet  pre-defined  criteria  for  safe 
flight.  Our  earlier  work  on  inspection  is  directly  rele¬ 
vant  to  the  present  study:  Patel,  et  al.  (1993)  investi¬ 
gated  specific  design  issues  relevant  to  inspection  using 
workcards. 

8.1.2  Workcard  Design  Issues 

Patel,  et  al,  (1993)  found  that  usable  documentation 
must  embrace  the  following  factors:  information  read¬ 
ability,  information  content,  information  organization, 
and  physical  handling  and  environmental  factors.  In¬ 
formation  readability  issues  are  concerned  with  the 
documentation’s  typographic  layout,  as  well  as  con¬ 
ventions  concerning  sentences,  words,  and  letters.  In- 
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formation  content  involves  what  information  to  give, 
how  to  give  it,  and  in  what  order.  Documentation  must 
be  appropriate,  accurate,  complete,  and  easily  compre¬ 
hensible.  Information  organization  deals  with  the  clas¬ 
sification  and  differentiation  of  directive  information 
and  other  information  such  as  notes  and  warnings.  The 
structure  of  directive  information  should  be  broken 
down  into  the  command  verb,  the  action  qualifier,  and 
the  object  of  the  action  (Inaba,  1991).  Patel,  et  al. 

(1993),  in  their  study  of  A-  and  C-checks,  pointed  out 
that  tasks  should  be  listed  in  the  natural  sequence  most 
inspectors  use  during  a  check.  Finally,  the  workcard 
must  be  physically  suitable  for  the  tasks  and  the  envi¬ 
ronment.  Inspectors  should  be  able  to  carry  workcards 
with  them  while  they  perform  tasks,  without  the  work- 
cards  hindering  task  performance.  Workcards  should 
be  resilient  to  all  types  of  weather  and  to  dirt  and  oil 
because  inspections  are  performed  under  a  variety  of 
adverse  conditions. 

Patel,  et  al.  (1993,  p.  13-16)  developed  a  set  of  guide¬ 
lines  for  designing  documentation  for  aircraft  inspec¬ 
tion  tasks.  Using  these  guidelines  to  redesign 
workcards,  they  found  significant  improvements  in  in¬ 
spectors'  and  mechanics’  ratings  of  redesigned  work- 
cards  when  compared  with  old  workcards.  These 
researchers  also  observed  that,  for  A-check  workcards, 
the  sequence  of  tasks  did  not  match  the  sequence  me¬ 
chanics  typically  follow  to  perform  checks.  There  is 
some  variability  in  the  ways  mechanics  and  inspectors 
sequence  their  tasks  throughout  a  check,  and  the  num¬ 
ber  of  sign-offs  varies  across  tasks.  These  findings 
demonstrate  the  need  for  investigation  of  issues  related 
to  workcard  task  sequence  and  the  optimal  number  of 
sign-offs. 

8.1.3  Purpose  of  Project 

This  project’s  original  aim  was  to  undertake  an  ex¬ 
perimental  evaluation  of  checklist  reliability.  The  fac¬ 
tors  of  interest  were  the  grouping  of  tasks  and  the 
number  of  sign-offs  required.  Different  workcard  for¬ 
mats  were  to  be  designed  for  less-detailed,  frequently 
performed  checks  such  as  low-level  and  A-checks. 
Possible  formats  would  have  included  workcards  with 
sign-offs  after  each  step,  with  sign-offs  only  after  the 
most  salient  items,  and  two-level  checklists  providing 
more-detailed  information  for  less-experienced  me¬ 
chanics.  The  methodology  of  this  project  changed  from 
an  off-line  experiment  to  a  field  study  at  the  request  of 
our  airline  partner  and  after  our  observation  of  mechan¬ 
ics  performing  these  checks. 

The  task  analysis  described  in  the  next  section  shows 
that  present  workcards  do  not  provide  mechanics  and 


inspectors  with  the  most  useful  information.  Although 
mechanics  and  inspectors  do  read  workcards  for 
changes,  they  do  not  continually  use  workcards  as  they 
perform  the  checks.  They  are  highly  practiced  in  their 
tasks,  and  the  fact  that  checks  are  repetitive  makes  it 
difficult  to  ensure  that  all  tasks  are  performed  to  the 
same  level  each  time.  Job  aids  or  redesign  of  workcards 
may  help  achieve  the  reliability  required  in  aircraft  in¬ 
spection.  This  is  why  we  changed  the  project’s  aims  to 
determining  how  mechanics  use  workcards,  why  me¬ 
chanics  do  not  use  workcards  continually  during  some 
checks,  the  possible  effects  of  mechanics  not  using 
workcards,  and  how  to  make  workcards  meet  checklist 
objectives  Degani  and  Wiener  (1990;  1993)  defined. 


The  project’s  first  objective  was  to  determine  how 
mechanics  actually  use  workcards  during  fre¬ 
quently  performed  checks.  We  needed  to  study  work- 
card  usage  on  the  hangar  floor  to  establish  the  degree 
that  workcards  meet  Degani  and  Wiener’s  checklist 
objectives.  A  task  analysis  of  a  system  is  the  foundation 
of  any  human  factors  investigation  (Drury,  Prabhu,  and 
Gramopadhye,  1990). 

8.2.1  Task  Analysis 

Our  study  of  mechanics'  current  use  of  workcards  dur¬ 
ing  checks  consisted  of  videotaping  and  observing  me¬ 
chanics  performing  thi'ee  levels  of  checks,  as  well  as 
interviews  and  workcard  evaluations.  We  made  no 
videotape  without  the  mechanics’  permission.  Video¬ 
taping  is  an  unintrusive  way  to  gain  accurate  informa¬ 
tion  on  how  a  mechanic  normally  performs  a  check. 

The  specific  checks  we  studied  were  A-checks  and  two 
less-detailed  checks:  lower-level  check  1  (least  com¬ 
prehensive)  and  lower-level  check  2  (more  compre¬ 
hensive,  but  less  than  an  A-check).  Our  activity  during 
our  first  two  trips  to  a  hangar  consisted  of  following 
mechanics  as  they  performed  the  check.  An  observer 
asked  questions  to  gain  a  basic  understanding  of  each 
check  for  various  types  of  equipment.  The  primary  data 
we  gathered  from  videotapes  were  the  sequence  of 
tasks  a  mechanic  performed,  the  number  of  times  a  me¬ 
chanic  referred  to  the  workcard,  and  the  approximate 
number  of  times  a  mechanic  was  interrupted.  After  me¬ 
chanics  finished  a  check,  we  interviewed  them,  often 
while  they  viewed  the  videotape  of  their  inspection  ac¬ 
tivity.  We  also  questioned  supervisors  and  lead  me¬ 
chanics  about  the  workcards’  usefulness  and  asked  for 
their  suggestions  for  change.  In  order  to  gain  opinions 
from  an  adequate  number  of  mechanics,  we  distributed 
evaluations  on  both  the  workcards  and  the  subsequently 
developed  job  aids  at  one  maintenance  base.  We  pres- 
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ent  results  of  videotaping,  interviews,  and  workcard 
evaluations  so  that  readers  may  develop  an  understand¬ 
ing  of  workcards’  usefulness  for  frequently  performed, 
repetitive  checks. 

8.2.1 .1  Mechanics'  Attitudes  Towards  the  Workcards 

Responses  to  interviews  and  workcard  evaluations  we 
distributed  to  mechanics  provided  many  interesting  in¬ 
sights.  Perhaps  the  most  important  finding  is  that  me¬ 
chanics  use  individual  methods  and  skills  to  complete 
checks.  Lock  and  Strutt  (1985),  in  their  study  of  the  re¬ 
liability  of  inspections  in  British  aviation,  had  similar 
findings.  The  implication  of  this  finding  is  that  it  is  dif¬ 
ficult  to  establish  reliability  of  checks  because  mechan¬ 
ics  do  not  value  the  standard  workcard, 

Workcard  evaluation  results  are  presented  in  Appendix 
8-A.  Question  5  in  Section  II  showed  that  some  me¬ 
chanics  do  not  usually  refer  to  a  workcard  during  a 
check.  About  half  responded  that  they  perform  a  par¬ 
ticular  check  in  the  same  sequence  each  time  they  per¬ 
form  the  check.  Most  indicated  that  they  sequence  tasks 
based  on  locations  on  the  airplane;  they  start  with  the 
nose  and  work  around  the  aircraft  to  check  for  discrep¬ 
ancies.  If  a  check  is  assigned  to  two  people,  tasks  are 
typically  divided  logically,  e.g.,  into  exterior  and  inte¬ 
rior  tasks.  The  exterior  is  usually  checked  before  the 
interior.  Some  mechanics  sequence  tasks  by  difficulty 
and/or  the  probability  of  finding  a  discrepancy  that 
must  be  fixed.  If  they  need  assistance,  they  request  a 
“floater”  to  help  them.  Appendix  8-B  shows  mechanics' 
ratings  of  task  difficulty  and  the  probability  of  finding  a 
discrepancy  for  B-737  lower-level  2  checks.  Tires  and 
brakes  generate  the  most  concern  because  of  the  time 
required  to  change  them  when  a  discrepancy  is  identi¬ 
fied. 

Although  workcard  evaluation  results  indicate  that  me¬ 
chanics  find  workcards  useful,  interviews  with  and  ob¬ 
servations  of  mechanics  performing  checks  indicate 
that  workcards  are  not  always  used  as  intended.  Many 
mechanics  view  workcards  as  guides  only  for  inexperi¬ 
enced  workers  who  may  refer  to  it  during  a  check: 
checks  become  routine  and  easily  memorized.  Also, 
mechanics  typically  check  more  items  than  the  work- 
card  requires  because  of  their  conscientious  natures. 
Most  mechanics  feel  that  they  only  need  to  refer  to  a 
workcard  for  interim  changes  before  performing  a 
check.  When  mechanics  find  a  discrepancy  during  a 
check,  most  state  that  they  make  a  note  to  fix  the  dis¬ 
crepancy  after  they  finish  the  check.  However,  the  ob¬ 
server  rarely  saw  notetaking,  with  the  exception  of  one 
mechanic.  This  could  be  because  some  mechanics  do 
not  carry  workcards  continuously  while  performing  a 


check.  After  completing  a  check,  mechanics  return  to 
the  workcard  to  sign-off  the  tasks.  The  question  re¬ 
maining  is,  if  mechanics  do  not  use  the  workcard  to  se¬ 
quence  tasks  for  a  check,  what  are  the  reasons  for  this 
and  how  do  they  sequence  the  required  tasks? 

8.2.1 .2  Content  of  the  Check 

One  reason  mechanics  rarely  use  the  workcard  while 
performing  these  checks  is  that  the  lower-level  and  A- 
checks  are  repetitive  and  frequent.  Most  of  these  me¬ 
chanics  perform  fifteen  lower-level  2  checks  and  five 
A-checks  every  month.  They  have  done  these  checks  at 
this  maintenance  base  for  an  average  of  9  years  (this  re¬ 
sult  came  from  the  workcard  evaluations  shown  as  Ap¬ 
pendix  8-F).  Furthermore,  checks  for  various  kinds  of 
equipment  are  similar,  with  only  a  few,  possibly  impor¬ 
tant,  differences.  Mechanics  easily  memorize  the 
checks  and  believe  they  do  not  need  workcards  as  port¬ 
able  job  aids. 

8.2.1 .3  Task  and  Environmental  Factors 

Lower-level  and  A-checks  are  mobile:  their  tasks  are 
located  throughout  an  airplane’s  exterior  and  interior. 
Mechanics  walk  around  a  plane  to  check  for  defects, 
bending,  kneeling,  or  reaching  into  an  access  panel. 
These  movements  are  not  conducive  for  carrying  an  8.5 
X  1 1  inch  workcard  that  a  mechanic  can  refer  to,  make 
notes  on,  and  sign-off  tasks.  In  addition,  many  line 
checks  are  performed  outside  in  a  variety  of  weather 
conditions  such  as  wind,  cold,  rain,  and/or  snow.  Carry¬ 
ing  a  paper  workcard  and  writing  on  it  is  even  less 
practical  in  these  circumstances. 

8.2.1 .4  Sequence  of  Tasks 

Patel,  et  al.  (1993)  found  that  mechanics'  ordering  of 
tasks  for  an  A-check  did  not  match  the  workcard’ s  or¬ 
der.  In  the  current  study,  mechanics  also  rarely  per¬ 
formed  tasks  in  the  order  listed  on  the  workcard.  In  a 
second  workcard  evaluation,  mechanics  were  asked  to 
order  tasks  of  a  B-737  lower-level  2  check  in  the  se¬ 
quence  they  normally  complete  the  check.  Appendix  8- 
C  presents  results  of  this  workcard  evaluation.  No  me¬ 
chanic  provided  the  sequence  given  in  the  workcard. 
Subjects  1  and  2  have  an  additional  column  in  their  ta¬ 
bles  since  they  were  videotaped.  In  addition  to  se¬ 
quence  data  from  workcard  evaluations,  transcript 
analyses  from  videotapes  of  subjects  performing 
checks  show  that  mechanics  do  not  use  workcards  to 
sequence  their  tasks.  Tasks  that  are  difficult  to  observe 
directly  are  indicated  by  asterisks  in  Appendix  8-C. 
This  does  not  indicate  that  tasks  were  not  performed, 
only  that  the  observer  could  not  see  them  on  the 
videotape. 
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Workcard  evaluations  and  videotapes  indicate  that  me¬ 
chanics  tend  to  sequence  tasks  by  spatial  cues  on  the 
airplane,  associating  a  specific  area  on  the  aircraft  with 
all  checks  for  that  area.  For  example,  at  the  right  main 
landing  gear,  a  mechanic  checks  tires  for  serviceability, 
checks  the  tire  pressure,  checks  the  tie  bolts,  cleans  the 
strut  piston,  cleans  the  downlock  viewer  and  indicator, 
and  checks  the  brakes.  All  these  tasks  are  performed  at 
the  right  main  landing  gear  before  the  mechanic  moves 
to  another  area.  The  workcard ’s  functional  organiza¬ 
tion,  however,  asks  a  mechanic  to  check  all  tires  for 
serviceability  before  moving  to  another  sign-off  task. 
This  would  require  a  mechanic  to  walk  around  the  nose 
landing  gear,  the  right  main  landing  gear,  and  the  left 
main  landing  gear  and  then  to  revisit  the  same  locations 
to  check  the  tire  pressures.  The  workcard  sequence 
does  not  reflect  the  way  most  people  work.  Tasks  such 
as  "Check  fuselage,  empennage,  and  wings  for  obvious 
damage  or  irregularities  as  viewed  from  the  ground" 
demonstrate  this  point  even  more  dramatically.  A  me¬ 
chanic  does  not  check  the  entire  fuselage  for  discrep¬ 
ancies  at  once;  instead,  he  or  she  checks  the  fuselage 
while  working  around  the  aircraft  performing  other 
checks.  This  is  demonstrated  by  the  numerous  times 
mechanics  being  videotaped  checked  the  fuselage;  they 
often  cover  the  same  area  more  than  once  and  re-visit 
the  same  task  numerous  times  (see  Appendix  8-C). 

Mechanics  organize  tasks  by  spatial  cues,  not  by  work- 
cards’  functional  order,  because  areas  to  be  inspected 
are  very  large.  Humans  optimize  their  use  of  time  by 
minimizing  the  distance  to  be  traveled.  By  checking 
everything  in  a  particular  aircraft  area  before  moving  to 
an  adjacent  area,  a  mechanic  saves  significant  time  and 
energy  compared  with  that  necessary  to  walk  around 
the  airplane  as  many  times  as  would  be  necessary  to 
check  everything  by  functions.  Using  spatial  cues,  in¬ 
stead  of  functional  locations,  reduces  the  number  of 
things  a  mechanic  must  remember,  hence  reducing  his 
or  her  mental  workload. 

There  is  a  mismatch  between  the  tool  provided  for  the 
job  (workcard)  and  mechanics'  natural  way  of  working. 
Such  a  mismatch  can  be  addressed  either  by  altering 
the  tool  or  by  altering  the  way  of  working.  The  altera¬ 
tion  chosen  depends  ultimately  upon  what  system  reli¬ 
ability  is  obtainable. 

8o2»2  NoEi-CompMaiinice  in  Usmg  Workcards 

Our  observations  from  other  airlines  during  previous 
projects  confirm  this  project’s  findings.  For  rarely  per¬ 
formed  tasks,  such  as  most  C-  and  D-checks,  inspectors 
use  workcards  to  perform  the  check.  Mechanics  do  not 
use  workcards  for  frequently  performed  checks,  i.e.,  A- 


checks  and  below.  They  have  memorized  these  checks, 
"gaining  a  feel  for  items  to  check"  through  frequent 
repetition.  One  of  the  problems  with  this  is  that  me¬ 
chanics  may  not  receive  feedback  on  the  accuracy  of 
their  judgments  since  problems  rarely  occur.  Also, 
since  workcards  are  not  physically  compatible  with  the 
environment  and  the  tasks,  even  inexperienced  mechan¬ 
ics  who  want  to  use  workcards  have  difficulty  doing  so. 
Finally,  the  functional  sequence  of  tasks  on  workcards 
does  not  match  the  way  people  sequence  tasks  distrib¬ 
uted  over  large  areas.  Tasks  with  only  one  sign-off  for 
a  particular  function  are  often  distributed  over  large  ar¬ 
eas  of  an  aircraft,  e.g.,  check  the  tire  pressure  of  the 
main  landing  gear  tires,  and  are  performed  as  a  me¬ 
chanic  reaches  the  area.  Since  mechanics  tend  to  sign- 
off  all  tasks  when  the  entire  check  is  complete,  tasks 
that  are  not  completed  sequentially  should  have  sepa¬ 
rate  sign-offs.  We  conclude  that  present  workcards  do 
not  provide  useful  information  for  mechanics  and,  con¬ 
sequently,  do  not  meet  the  checklist  objectives  Degani 
and  Wiener  (1990;  1993)  defined. 

8.2.3  Relatiomship  Betweesi  Workcard  aed  Check¬ 
list  Objectives 

To  review,  the  objectives  of  a  checklist  are  to  aid  the 
user  in  recalling  procedures,  to  outline  a  convenient  se¬ 
quence  for  motor  movements  and  eye  fixations,  to  al¬ 
low  mutual  supervision  within  a  crew,  to  distribute 
tasks  among  crew  members,  and  to  function  as  a  qual¬ 
ity  control  tool  for  management  and  government  regu¬ 
lators  (Degani  and  Wiener,  1990;  1993).  Since  present 
workcards  do  not  provide  a  convenient  sequence  for 
motor  movements  and  eye  fixations,  they  are  not  used 
continuously  during  checks.  The  workcards  do  not  aid 
the  user  to  recall  procedures.  The  present  workcards 
cannot  be  used  conveniently  to  distribute  tasks  among 
mechanics  because  many  sign-offs  are  not  separated. 
The  practice  of  signing  off  tasks  at  the  end  of  the 
checks  diminishes  the  workcards’  ability  to  serve  as  a 
quality  control  tool.  A  job  aid  needs  to  be  designed  that 
meets  checklist  objectives  listed  above  and  that  ac¬ 
commodates  mechanics'  different  work  methods.  Me¬ 
chanics  working  for  many  different  airlines  would  use 
such  a  job  aid. 


That  the  present  system  appears  to  be  working  is 
demonstrated  by  high  reliability,  i.e.,  accidents  are 
extremely  rare.  However,  mechanics’  workcard  use  is 


reduced  because  the  job  aids  do  not  match  their  needs 


and  individual  work  methods.  The  danger  of  not  using 
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workcards  during  a  check  is  that  a  mechanic  must  then 
rely  solely  on  his  or  her  memory.  If  a  mechanic  were  to 
become  distracted,  he  or  she  could  forget  to  perform  a 
check,  yet  automatically  sign  it  off  because  he  or  she 
has  performed  the  check  so  many  times  correctly.  A 
mechanic’s  confusion  with  similar  checks  and  other 
aircraft  may  result  in  him  or  her  substituting  a  required 
task  with  a  task  appropriate  for  another  check  or  air¬ 
craft. 

Our  observations  from  other  airlines  indicate  that 
similar  patterns  in  workcard  usage  exist  throughout  the 
industry.  It  is  worthwhile  to  place  our  findings  in  a 
broader  context  by  analyzing  similar  errors  reported 
elsewhere.  The  following  examples  of  errors  relating  to 
these  issues  are  taken  from  NASA’s  Aviation  Safety 
Reporting  System  (ASRS).  These  voluntary  reports  are 
subject  to  reporting  biases,  and  no  airline  is  named  in 
these  reports. 

The  following  excerpts  from  ASRS’  reports  illustrate 
the  importance  of  workcards  meeting  checklist  goals. 
They  also  illustrate  other  problems,  such  as  the  speed- 
accuracy  tradeoff  and  poor  training,  but  all  have  a 
common  contributing  cause  of  mechanics’  not  follow¬ 
ing  procedures  specified  on  the  workcard. 

*  I  had  just  completed  an  outside 
service  inspection... when  an  FAA 
inspector  pointed  out  that  I  had 
failed  to  check  for  water  in  the  fuel 
tanks  and  had  missed  a  couple  of  un¬ 
readable  placards  but  had  signed  off 
blocks  saying  I  had  checked  these 
items.  Both  were  inadvertent  over¬ 
sights,  were  not  deliberate,  and  did 
not  cause  any  significant  unsafe 
conditions.  The  problem  arose  be¬ 
cause  I  was  in  a  hurry  to  get  the  job 
done.  Also,  in  the  2  years  that  I  have 
worked  on  these  aircraft,  I  have 
never  heard  of  any  mechanics  find¬ 
ing  water  in  the  fuel  tanks.  I  have 
corrected  the  situation  by  slowing 
down  and  paying  attention  to  the 
checklist  and  my  actions. 

*  While  performing  an  A 
check,. ..one  of  my  coworkers,  Y, 
pencil-whipped  the  aircraft  landing 
gear  and  flap  lube.  I  had  been 
working  the  engines  all  night  and 
know  that  the  flaps  had  not  been  ex¬ 
tended  for  lubing. 


*  I  did  not  perform  a  pitot  static  leak 
check  on  the  altimeter  system  after 
altimeter  replacement....!  was  at  fault 
because  I  was  unaware  that  the 
maintenance  manual  had  been  re¬ 
vised  to  reflect  this  change. 

*  Due  to  an  oversight,  not  having  the 
sign-off  document  immediately 
available,  I  did  not  document  the 
company  form  that  I  had  complied 
with  XXXX,  a  visual  inspection  of 
the  cargo  door  prior  to  takeoff 

*  I  feel  my  actions  may  well  be  the 
cause  of  the  gear  failure  due  to  im¬ 
proper  reassembly  of  the  uplock  ac¬ 
tivator,  and  failure  to  follow  proper 
procedures.  In  addition,  I  made  sev¬ 
eral  mistakes  in  following  the  proper 
procedures,  as  called  for  by  com¬ 
pany  maintenance  manuals.  I  failed 
to  enter  a  discrepancy  on  a  me¬ 
chanic’s  discrepancy  list.  I  did  not 
use  proper  maintenance  manual  re¬ 
views.  I  did  not  perform  a  gear  re¬ 
traction  following  reassembly  of  the 
activator. 

These  reports  all  illustrate  errors  that  could  be  attrib¬ 
uted  to  not  using  or  not  complying  with  workcards  or 
maintenance  manuals.  The  first  two  reports  provide  ex¬ 
amples  of  workers  signing-off  tasks  they  did  not  per¬ 
form.  The  example  of  a  mechanic  not  performing  a  fuel 
tank  sump  check  demonstrates  one  of  the  effects  of  ex¬ 
perience.  Since  the  mechanic  does  not  expect  to  find  a 
problem,  the  check  is  not  taken  seriously.  The  report  of 
an  inspector  or  mechanic  being  unaware  of  a  mainte¬ 
nance  manual  revision  is  an  example  of  a  failure  to  read 
interim  changes.  The  fourth  account  states  that  the  re¬ 
porter  did  not  have  the  workcard  immediately  avail¬ 
able,  probably  because  the  workcard  was  incompatible 
with  the  task  and  environment.  The  last  report  provides 
another  example  of  a  mechanic  not  complying  with 
proper  procedures.  This  could  be  attributed  to  numer¬ 
ous  factors  such  as  training,  the  mechanic’s  attitudes, 
time  constraints,  and  environmental  factors  that  make 
using  the  maintenance  manual  either  difficult  or  incon¬ 
venient. 

*  After  servicing  #1  engine  and 
while  servicing  #3  I  was  distracted 
by  another  crew  member  standing 
below  my  servicing  buggy.  He 
wanted  me  to  check  something  else 
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on  the  aircraft  and  after  doing  so  I 
returned  to  my  servicing  buggy,  still 
thinking  that  I  had  finished  #3  en¬ 
gine.  I  moved  on  to  another  aircraft. 
This  aircraft  took  off  and  during  the 
first  part  of  the  flight  the  crew  noted 
the  #3  engine  oil  level  falling  and 
then  stabilizing  at  an  acceptable 
level.  Upon  landing  the  crew  called 
maintenance,  who  found  the  #3  en¬ 
gine  oil  service  door  missing,  along 
with  the  oil  cap. 

*  During  the  reassembly  procedure 
the  screws  were  not  installed  in  the 
panel.  I  was  called  away  by  a  co¬ 
worker  and  foreman  to  help  on  an¬ 
other  problem  on  the  aircraft.  Then  a 
push  to  get  the  aircraft  on  line  oc¬ 
curred.. ..The  aircraft  was  stopped  at 
its  next  destination;  the  panel  was 
found  missing. 

*  On  the  aircraft’s  right  wing  tail 
light  assembly,  I  removed  the  light 
assembly  to  change  the  top 
bulb.. ..Note:  On  removal  of  the  unit, 

I  had  laid  the  8  securing  screws  on 
top  of  the  wing.  Before  I  secured  the 
unit  into  the  wing  tip,  I  wanted  to  be 
sure  it  worked.  I  went  into  the 
cockpit  and  activated  the  lights.  I 
went  out  to  the  wing  tip  to  find  them 
working  properly  and  returned  to 
cockpit  to  shut  them  off,  as  the  lights 
would  be  blinding  while  securing  the 
unit.  After  shutting  lights  off  from 
cockpit,  I  stopped  for  3-4  minutes  to 
talk  to  a  mechanic  who  was  doing 
aircraft  interior  work.... After  leaving 
the  interior  of  the  aircraft,  I  was 
thinking  I  wanted  to  finish  all  exte¬ 
rior  work  quickly,  as  it  was  18  de¬ 
grees  F  with  the  wind  chill  factor.  A 
ladder  I  had  out  on  the  left  engine 
caught  my  eye  as  I  was  coming 
down  the  stairs.  I  was  running 
through  my  mind  items  I  had  to 
complete  to  get  inside  out  of  the 
weather.  With  the  wing  tip  light 
fixed,  all  I  had  to  do  was  put  the  lad¬ 
der  away  [without  securing  the 
screws]. 


These  errors  demonstrate  potential  negative  effects  of 
inattention  and  distractions.  Although  the  mechanics 
we  interviewed  all  strongly  stated  that  if  they  were  dis¬ 
tracted  they  would  not  need  to  make  a  note  to  remem¬ 
ber  which  tasks  to  complete,  most  research  in  human 
error  suggests  otherwise.  Reason  (1990)  developed  a 
human  error  model  that  particularly  considers  the  ef¬ 
fects  of  inattention. 

8.3.1  Applicable  Human  Error  Research 

Rasmussen  (1982)  models  human  performance  and  its 
interactions  with  a  possibly  unaccommodating  envi¬ 
ronment,  categorizing  it  on  the  basis  of  human  infor¬ 
mation  processing.  At  the  skill-based  (SB)  performance 
level,  people  perform  familiar,  routine  tasks  requiring 
little  attention.  Rule-based  (RB)  activities  involve  using 
established  rules  to  make  familiar  decisions  or  to  solve 
common  problems.  Knowledge-based  (KB)  perform¬ 
ance  is  employed  when  no  known  rules  are  available 
for  the  situation  and  a  person  must  resort  to  reasoning, 
to  mental  models,  and  to  high-order  cognitive  proc¬ 
esses  to  appraise  the  available  information,  to  assign 
goals,  and  to  develop  methods  for  achieving  them. 

Reason  (1990)  describes  two  cognitive  modes  for  dif¬ 
ferentiating  between  the  sequential  reasoning  used  for 
KB  tasks  and  the  automatic  control  used  for  SB  and 
RB  tasks.  The  attentional  mode  for  knowledge-based 
activities  requires  high  cognitive  effort  and  is  character¬ 
istic  of  the  decision-maker’s  low  level  of  experience 
with  the  problem  or  situation.  During  SB  and  RB  per¬ 
formance,  the  schematic  mode  involves  semi-automatic 
actions  with  few  or  no  attentional  checks.  A  person’s 
intentions  or  matching  conditions  in  the  environment 
activate  strongly  associated  groups  of  actions  called 
"schemata." 

Reason  writes,  "When  cognitive  operations  are  under- 
specified,  they  tend  to  default  to  contextually  appro¬ 
priate,  high-frequency  responses... or... the  more  often  a 
cognitive  routine  achieves  a  successful  outcome  in  re¬ 
lation  to  a  particular  context,  the  more  likely  it  is  to  re¬ 
appear  in  conditions  of  incomplete  specification" 

(1990,  p.  97).  In  other  words,  when  a  person  cannot 
define  all  aspects  of  a  situation,  he  or  she  resorts  to 
habitual  actions.  Incomplete  specification  of  a  situation 
can  be  attributed  to  a  combination  of  situational  factors 
and/or  a  person’s  lack  of  attention.  Errors  result  from 
activation  of  the  wrong  schemata  or  from  activating  the 
right  schemata  either  in  the  wrong  order  or  at  the  wrong 
time.  As  a  person  becomes  practiced  with  a  habitual 
task,  the  chances  of  activating  a  common,  yet  inappro¬ 
priate,  schemata  increase. 
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Errors  often  occur  in  "strong-but- wrong"  form,  i.e.,  be¬ 
havior  is  appropriate  to  past  circumstances  because  of 
lack  of  attention  to  changed  circumstances.  Skill-based 
performance  errors  occur  because  actions  at  this  level 
are  directed  by  schemata  most  active  when  an  atten- 
tional  check  is  omitted  or  mistimed.  Rule-based  per¬ 
formance  errors  are  usually  attributed  to  inappropriate 
associations  between  contextual  cues  and  previously 
applicable  rules.  Knowledge-based  performance  errors 
are  unpredictable  since  the  person  does  not  have  the 
knowledge  to  deal  with  the  unfamiliar  situation.  These 
errors  are  due  to  "bounded  rationality"  and  incomplete 
or  inaccurate  mental  models  (Reason,  1990). 

The  potential  skill-based  errors  is  particularly  impor¬ 
tant  for  repetitive  lower-level  and  A-checks.  Experi¬ 
enced  mechanics  quite  familiar  with  the  tasks  operate  at 
the  skill-based  level  when  they  move  between  tasks 
within  a  check.  When  an  attentional  check  is  omitted, 
the  mechanic  does  not  specifically  note  where  he  or  she 
is  in  the  task  sequence.  The  mechanic  then  can  easily 
be  "captured"  by  a  schema  or  another  task  that  he  or 
she  frequently  would  perform  in  that  situation,  even  if 
the  mechanic’s  intentions  call  for  a  different  action.  For 
example,  an  attentional  check  can  be  omitted  because 
of  an  external  interruption  such  as  another  crew  mem¬ 
ber  asking  the  mechanic  to  check  something.  The  dis¬ 
traction  could  be  internal,  e.g.,  the  mechanic  worrying 
about  other  tasks,  the  weather,  even  time  pressure. 

Mechanics  may  use  rules  to  determine  if  an  indication 
is  a  discrepancy.  One  objective  of  workcards  and 
maintenance  manuals  is  to  externalize  rules  so  the  me¬ 
chanic  does  not  need  to  remember  them.  For  example, 
the  workcard  gives  the  acceptable  range  of  tire  pres¬ 
sure.  If  the  mechanic  does  not  use  the  workcard,  the 
potential  for  rule-based  errors  rises  since  the  mechanic 
is  forced  to  rely  on  memory.  Rules  often  differ  among 
tasks  which  are  otherwise  similar,  e.g.,  different  tire 
pressures  are  acceptable  for  different  aircraft. 

Knowledge-based  errors  are  not  relevant  to  the  checks 
under  study  in  this  project.  As  mentioned,  lower-level 
and  A-checks  are  repetitive  and  familiar  for  these  me¬ 
chanics.  Knowledge-based  reasoning  rarely  occurs; 
when  it  does,  a  workcard  is  likely  to  be  of  little  assis¬ 
tance.  In  knowledge-based  situations,  maintenance 
manuals  and  a  mechanic’s  experience  and  knowledge 
are  the  best  resources.  The  goals  of  checklists  are  to 
assist  skill-based  and  rule-based  performance  and  to 
compel  mechanics  to  make  more  attentional  checks 
while  they  work  in  the  schematic  mode.  The  errors 
listed  in  the  next  section  are  associated  with  workcards’ 
failure  to  meet  objectives  for  checklists. 


8.3.2  Potential  Errors  Related  to  Workcards 

We  derived  the  following  potential  errors  after  consid¬ 
ering  Reason’s  theories  of  human  error  and  from  our 
study  of  workcard  usage.  We  made  our  predictions  of 
potential  types  of  errors  related  to  workcards  knowing 
that  mechanics  rarely  use  workcards,  that  they  sign-off 
all  tasks  at  the  end  of  the  check,  and  that  the  potential 
for  distractions  and  interruptions  is  high  as  they  per¬ 
form  these  checks.  The  first  three  kinds  of  errors  are 
omissions  related  to  skill-based  performance.  The  last 
category  is  related  to  rule-based  errors.  There  are  other 
kinds  of  potential  errors,  but  the  following  are  most 
relevant  to  findings  of  our  study  of  workcard  usage. 

8.3.2. 1  Omissions  Related  to  Interruptions 

Reason’s  (1990)  theories  predict  that  distractions  and 
interruptions  occurring  while  workers  perform  highly 
skilled,  familiar  tasks,  such  as  lower-level  and  A- 
checks,  are  particularly  critical.  When  the  mechanic  di¬ 
rects  attention  back  to  the  check,  he  or  she  may  not 
finish  a  task  or  fail  to  perform  a  task.  Since  checks  are 
performed  in  the  schematic  mode,  task  completion 
within  a  check  is  fairly  automatic.  A  mechanic  recovers 
from  most  interruptions  by  making  a  conscious  effort  to 
ensure  continuity.  Unless  the  mechanic  makes  an  effort 
to  recall  what  he  or  she  was  doing  when  interrupted  or 
distracted,  the  mechanic  can  continue  the  check  after 
being  interrupted  as  in  the  most  frequently  occurring 
circumstances.  Since  the  mechanic  has  previously 
completed  the  task  numerous  times,  he  or  she  may  hon¬ 
estly  believe  the  task  to  have  been  completed.  As  the 
ASRS’  examples  illustrate,  the  mechanic  may  never  di¬ 
rect  attention  back  to  the  task,  particularly  if  there  is 
time  pressure  to  complete  the  check.  After  an  interrup¬ 
tion,  the  mechanic  may  start  on  a  new  set  of  tasks  and 
never  return  to  his  or  her  original  mental  task  list.  Pos¬ 
sible  remedies  for  these  types  of  errors  include  the  fol¬ 
lowing: 

a)  Workcards  should  be  designed  to  be 
easy  for  workers  to  make  notes  on  or 
to  sign  off  complete  tasks 

b)  Mechanics  should  be  informed  of  ef¬ 
fects  of  interruptions  and  distrac¬ 
tions,  as  well  as  the  importance  of 
making  notes  about  incomplete 
tasks. 

We  need  to  consider  ways  to  combat  all  errors  frequent 
enough  to  be  captured  by  ASRS. 
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8.3.2.2  Omissions  Related  to  Workcard  Sequence 

Workcards  sequence  tasks  by  functions.  If  mechanics 
actually  followed  workcards’  sequences,  the  probability 
of  distraction  would  increase  as  they  constantly  moved 
around  the  aircraft  to  complete  functional  checks.  In 
turn,  this  would  increase  the  likelihood  of  an  omission 
associated  with  an  interruption  or  distraction.  Sign-offs 
for  some  tasks  are  not  separated,  although  the  tasks  are 
spatially  separated.  For  example,  there  is  a  single  sign- 
off  for  serviceability  of  both  right  and  left  main  landing 
tires.  However,  tires  are  checked  separately.  This 
workcard  sequence  of  tasks  may  increase  the  probabil¬ 
ity  of  a  mechanic  signing-off  the  task  after  checking 
one  side  of  the  main  landing  tires,  but  before  checking 
both  sides. 

8.3.2.3  Omissions  Related  to  Workcard  Non-Compliance 

Task  analysis  of  mechanics  performing  checks  revealed 
that  workcards’  functional  sequence  of  tasks  rarely 
matches  the  spatial  sequence  mechanics  use.  The  task 
analysis  also  predicted  and  revealed  that  mechanics 
rarely  use  workcards,  partly  because  they  do  not  match 
work  habits  and  partly  because  they  are  physically  in¬ 
compatible  with  the  tasks  and  environment.  Mechanics 
disregarding  the  task  sequence  on  a  workcard  rely  on 
memory  and  are  thus  more  likely  to  omit  a  task,  par¬ 
ticularly  one  they  perceive  as  unlikely  to  reveal  a  dis¬ 
crepancy.  Since  mechanics  assigned  to  frequent  checks 
generally  perform  them  on  a  number  of  different  air¬ 
craft,  they  may  unknowingly  confuse  checks,  e.g., 
substitute  a  task  from  a  different  check  or  aircraft. 
Workcards  help  them  recall  tasks  to  be  performed. 

Most  mechanics  decrease  the  chances  of  this  type  of  er¬ 
ror  by  performing  substantially  more  checks  than  the 
workcard  requires.  For  example,  a  mechanic  may  treat 
part  of  a  lower-level  check  as  the  equivalent  part  of  an 
A-check. 

A  lack  of  a  rigidly  performed  sequence  is  likely  to  in¬ 
duce  omission(s)  when  the  task  sequence  is  not  habitual 
and  requires  more  attention.  A  number  of  mechanics 
indicated  that  they  do  not  follow  the  same  task  se¬ 
quence  each  time  they  perform  a  check.  Also,  mechan¬ 
ics’  practice  of  signing-off  all  tasks  at  a  convenient 
break,  even  at  the  end  of  a  check,  instead  of  immedi¬ 
ately  after  completing  a  task,  increases  the  likelihood  of 
an  omission  when  a  mechanic  frequently  performs  the 
checks.  If  an  omission  is  possible  due  to  a  distraction, 
time  pressure,  or  some  other  reason,  the  mechanic 
signing-off  tasks  must  pay  careful  attention  to  each  one 
he  or  she  signs-off,  and  must  actually  recall  performing 
that  task  at  that  time.  Since  sign-offs  are  highly  repeti¬ 
tive  and  require  very  little  attention,  a  mechanic  could 


easily  assume  that  a  task  was  completed  because  it 
previously  was  always  completed. 

8.3.2.4  Rule-Based  Errors 

One  of  the  objectives  of  a  checklist  is  to  aid  users  to  re¬ 
call  procedures  (Degani  and  Wiener,  1990;  1993). 
Workcards  mainly  outline  tasks  to  be  performed;  they 
also  remind  mechanics  of  some  specification  limits, 
such  as  those  for  tire  pressures.  Other  specification 
limits  are  not  given  on  the  workcards,  so  one  recom¬ 
mendation  for  improvement  is  to  include  all  limits  on 
the  workcard.  If  a  mechanic  does  not  regularly  use  a 
workcard  throughout  a  check,  he  or  she  may  confuse 
specification  limits  among  airplanes. 

More  likely  causes  of  rule-based  errors  relate  to  the 
nature  of  a  check  and  the  high  experience  levels  of  me¬ 
chanics  performing  them.  Because  mechanics  are  fa¬ 
miliar  with  the  checks,  they  may  not  readily  recognize 
unusual  circumstances,  as  Reason  predicts.  Although 
experience  normally  assists  mechanics  by  directing 
their  attention  to  likely  locations  of  defects,  it  may  hin¬ 
der  them  when  circumstances  substantially  differ  from 
their  expectations.  As  Lock  and  Strutt  write,  'There  is  a 
danger  that  too  much  familiarity  with  a  particular  item 
could  lead  an  experienced  inspector  to  miss  a  signifi¬ 
cant  defect,  if  it  does  not  conform  to  the  expected  pat¬ 
tern  (condition)  or  expected  locations  which  are  fixed 
in  the  inspector’s  mental  model  of  the  aircraft  and  its 
pattern  of  deterioration"  (1985,  p.  6.5).  Paradoxically, 
mechanics’  high  level  of  experience  and  expertise  is 
one  of  the  greatest  challenges  we  face  in  developing  a 
job  aid  for  the  checks. 

8.3.3  The  Challenge  of  Developing  a  Job  Aid 

Task  analyses  performed  with  existing  workcards  re¬ 
vealed  potential  causes  of  error  as  checks  are  currently 
performed.  A  job  aid  needs  to  be  designed  that  reduces 
the  potential  for  errors  associated  with  workcards  in¬ 
compatible  with  mechanics’  work  habits  and  for  eiTors 
related  to  mechanics’  failure  to  use  workcards  through¬ 
out  a  check.  These  errors  all  stem  from  the  fact  that  the 
present  workcard  is  frankly  not  useful  for  mechanics. 
The  design  difficulty  is  compounded  by  the  fact  that 
highly  skilled,  well-trained,  and  experienced  mechanics 
view  workcards  as  guides  for  inexperienced  mechanics 
and  as  quality  control  tools. 

This  project’s  challenge  was  to  help  increase  the  reli¬ 
ability  of  an  already  reliable  system.  Mechanics'  work 
is  extremely  reliable  without  workcards.  Even  when 
mechanics  make  an  error,  they  rarely  receive  feedback. 
Due  to  the  redundancy  and  frequency  of  checks,  air¬ 
planes  normally  fly  without  incident.  However,  there 
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remains  a  slight  possibility  that  not  using  workcards 
during  the  check,  or  using  workcards  that  do  not  match 
work  methods,  could  result  in  an  error  with  adverse 
consequences.  Adding  to  the  challenge  is  the  fact  that 
as  mechanics’  experience  increases,  the  probability  they 
use  a  workcard  as  intended  decreases.  It  is  worthwhile 
to  explore  developing  a  job  aid  that  reduces  the  small 
probability  of  error  because  it  is  compatible  with  me¬ 
chanics’  work  habits  and  meets  Degani  and  Wiener’s 
checklist  objectives.  Any  increase  in  reliability  is  worth 
the  effort  in  an  industry  ciffecting  public  safety  as  di¬ 
rectly  as  airlines. 


The  proposed  job  aid  must  meet  individual  mechanic’s 
work  methods,  must  be  physically  compatible  with 
their  environment  and  tasks,  and  must  meet  guidelines 
for  workcard  design  Patel,  Prabhu,  and  Drury  (1992) 
developed.  Mechanics  are  more  likely  to  use  a  job  aid 
with  these  characteristics. 

8.4.1  The  Development  of  the  Job  Aid 

Observations  and  videotapes  of  checks  revealed  that 
the  task  sequence  differs  among  mechanics.  Even  the 
same  mechanic  performs  tasks  for  the  same  check  in  a 
different  sequence  on  different  nights.  These  findings 
suggest  that  the  job  aid  must  be  flexible  in  task  se¬ 
quencing  and  adaptable  to  different  circumstances. 

Most  mechanics  order  tasks  by  using  spatial  locations 
on  an  airplane.  Appendix  8-D  lists  grouped  tasks  of  a 
B-737  lower-level  2  check  commonly  occurring  se¬ 
quentially  within  a  check.  We  developed  this  list  after 
analyzing  the  videotaped  checks.  We  organized  tasks  in 
a  FROM/TO  chart  that  showed  the  number  of  times 
two  tasks  were  performed  sequentially.  We  follow  each 
task  in  Appendix  8-D  with  a  list  of  tasks  performed  se¬ 
quentially  to  the  first  task  for  a  group.  Groups  largely 
mirror  the  spatial  layout  of  tasks  on  the  aircraft.  Work- 
card  tasks  could  be  divided  into  the  spatial  areas  in 
which  mechanics  perform  a  group  of  checks,  as  re¬ 
vealed  by  sequential  analysis. 

The  proposed  job  aid  organizes  tasks  spatially  by  list¬ 
ing  all  tasks  for  a  particular  area  of  the  aircraft  on  one 


pocket-sized  card.  The  cards  are  laminated  and  placed 
on  a  ring  so  that  a  mechanic  easily  can  change  the  order 
of  cards.  Figure  8.1  (next  page)  shows  the  fi-ont  page  of 
the  cards.  Dividing  tasks  by  area  into  small  cards  al¬ 
lows  a  mechanic  to  sequence  areas  according  to  his  or 
her  individual  work  habits.  Tasks  are  organized  with 
the  spatial  layout  most  mechanics  prefer.  A  mechanic 
can  use  a  grease  pencil  to  note  discrepancies,  inter¬ 
rupted  tasks,  or  sign-off  tasks  completed.  Notes  can 
then  be  copied  onto  reports  or  wiped  off  the  job  aid 
when  the  check  is  complete.  The  job  aid  cards  are  de¬ 
signed  to  have  a  bar  code  on  each  card  so  that  a  future 
scanning  system  could  check  which  cards  had  been 
completed  or  to  match  cards  with  bar  codes  located  on 
the  aircraft.  This  feature  was  removed  after  initial  de¬ 
sign  and  is  not  used  in  the  current  evaluation. 

Job  aids  were  designed  for  both  lower-level  checks  and 
for  A-checks  on  three  fleets  of  aircraft.  The  workcards’ 
design  follows  Patel,  et  al.'s  (1992)  guidelines  for  in¬ 
formation  readability,  information  content,  information 
organization,  and  physical  handling  and  environmental 
factors.  Some  guidelines  were  particularly  important 
for  this  job  aid. 

The  guidelines  for  information  content  recommend  that 
“information  provided  should  be  supportive  of  the  in¬ 
spector’s  personal  goal  to  read  quickly  and  also  under¬ 
stand  the  information,  to  ensure  its  usage  and  eliminate 
personal  biases”  (Patel,  et  al.,  1992,  p.l4).  We  ac¬ 
complished  this  in  the  job  aid’s  design  by  meeting  other 
guidelines  such  as  the  following: 

Resort  to  use  of  primary  typographic  spatial 
cues  like  vertical  spacing,  lateral  positioning, 
paragraphing  and  heading  positioning  as  far 
as  possible;  if  space  usage  is  premium,  then 
resort  to  use  of  secondary  cueings,  e.g.,  bold¬ 
facing,  italics,  underlining,  color  coding  and 
capital  cueing  in  a  decreasing  order  of  prefer¬ 
ence 
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Distinguish  between  directive  infor¬ 
mation,  reference  information, 
warnings,  cautions,  notes,  proce¬ 
dures  and  methods 

Directive  information  should  be  bro¬ 
ken  into  the  command  verb  (e.g., 
check),  the  objects  (e.g.,  valves,  hy¬ 
draulic  lines)  and  the  action  qualifi¬ 
ers  (e.g.,  for  wear,  frays).  Use  a 
consistent  typographic  layout 
throughout  the  document 

[The  content]  should  have  certain 
consistent  and  connmon  elements  to 
foster  generalizations  across  contexts 
(Patel,  et  al.,  1992,  pp.  13-15). 

Each  workcard’s  heading  refers  to  a  spatial  location  on 
the  aircraft  combined  with  a  functional  description, 
e.g.,  right  main  landing  tires,  right  forward  fuselage, 
flight  deck,  right  CSD  oil.  We  capitalized  the  headings 
and  centered  them  on  the  top  of  each  workcard.  Each 
heading’s  color  indicates  where  the  group  of  tasks 
listed  on  the  workcard  is  located  on  the  aircraft,  e.g., 
green  indicates  radome  and  forward  fuselage.  Color¬ 
coding  makes  sorting  cards  by  aircraft  areas  easier: 
mechanics  can  arrange  cards  in  their  preferred  se¬ 
quence  quickly.  Tasks  to  be  performed  are  left- 
justified.  Cautions  are  indented  and  bold.  Notes  are  in¬ 
dented  from  the  cautions  and  presented  in  a  smaller 
font  (see  Figure  8.2,  next  page).  Each  task  is  numbered 
on  the  workcard  and  separated  from  other  tasks  with 
blank  lines.  This  arrangement  makes  it  easier  for  me¬ 
chanics  to  distinguish  among  tasks  and  to  mark  com¬ 
pleted  tasks  with  a  grease  pencil.  The  command  verb 
immediately  follows  the  number;  it  is  followed  by  the 
object  and  the  action  qualifiers,  as  in  the  following  ex¬ 
ample: 

1)  Check:  forward  lavatory  for  general  appearance 
and  condition. 

The  command  verb  and  the  object  are  bold  because 
mechanics  already  know  the  action  qualifier  and  simply 
need  a  reminder  of  the  task  to  be  performed.  Some  me¬ 
chanics  suggested  listing  only  the  object  to  be  checked 
on  the  workcards.  We  could  not  investigate  this  idea  in 
this  project  because  regulations  do  not  allow  work- 
cards’  content  to  be  changed.  The  typographic  layout 
and  general  content  is  consistent  throughout  workcards 
for  all  checks,  ensuring  consistency  for  mechanics. 

The  following  are  the  organizational  issues  and  physi¬ 
cal  handling/environment  factors  we  considered  perti¬ 
nent  to  the  design  of  the  job  aid: 


Task  information  should  be  or¬ 
dered/sequenced  in  the  natural  order 
most  inspectors  would  perform  the 
tasks 

The  page  should  act  as  a  naturally 
occurring  information  module 

The  workcard’s  pages  should  be  a 
handy  size 

If  use  of  a  workcard  demands  expo¬ 
sure  to  environmental  agents  like 
wind,  rain,  snow  or  even  harsh  and 
oily  floor  conditions,  we  should  take 
adequate  precautions  to  avoid  ex¬ 
cessive  degradation"  (Patel,  et  al., 

1992,  p.  16). 

One  of  the  primary  goals  of  our  job  aid  is  to  meet  the 
guideline  concerning  the  order  of  task  information. 
Patel,  et  al.  (1992)  ordered  tasks  in  an  A-check  by 
finding  the  most  common  sequence  among  mechanics 
they  surveyed.  For  our  study,  we  took  an  approach 
based  upon  groups  of  tasks  that  mechanics  perform  se¬ 
quentially.  We  then  listed  each  group  of  tasks  on  one 
card  (for  an  example,  see  Figure  8.3,  page  139)  so  that 
workcards  act  cis  naturally  occurring  information  mod¬ 
ules.  Since  mechanics  can  arrange  the  groups  of  tasks 
in  any  order  they  choose;  our  job  aid  provides  a  natural 
sequence  to  al!  mechanics,  not  to  most  mechanics. 

Further,  the  pocket-sized  cards  leave  mechanics’  hands 
free,  when  necessary.  The  cards  are  laminated  to  pro¬ 
tect  them  against  environmental  agents  and  to  provide  a 
better  writing  surface  than  paper  (see  Figure  8.4,  page 
140). 

Although  we  encourage  mechanics  to  make  notes  on 
the  job  aids  and  to  check  tasks  completed,  the  job  aid 
does  not  replace  workcards’  sign-off  sheets.  The  first 
card  of  the  job  aid  explains  what  the  job  aid  is  and  in¬ 
structs  the  mechanic  to  read  interim  changes  included 
in  the  workcard  and  to  sign-off  tasks  on  the  workcard. 
The  second  card  shows  the  headings’  colors  and  asso¬ 
ciates  colors  with  areas  of  the  aircraft.  These  features 
help  meet  the  checklist  objectives  and,  consequently, 
reduce  the  potential  for  error. 
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8.4.2  Does  The  Job  Aid  Meet  Checklist  Objec¬ 
tives? 

To  review,  the  objectives  of  a  checklist  are  to  aid  the 
user  in  recalling  procedures,  to  outline  a  convenient  se¬ 
quence  for  motor  movements  and  eye  fixations,  to  al¬ 
low  mutual  supervision  in  a  crew,  to  distribute  tasks 
among  crew  members,  and  to  function  as  a  quality 
control  tool  for  management  and  government  regula¬ 
tors  (Degani  and  Wiener,  1990;  1993).  Dividing  tasks 
spatially  in  small  cards  affords  a  mechanic  the  flexibil¬ 
ity  to  sequence  areas  according  to  his  or  her  individual 
work  habits  while  also  organizing  the  tasks  spatially. 
The  job  aid  provides  a  convenient  sequence  for  motor 
movements  within  an  area  while  allowing  a  mechanic 
to  determine  the  most  convenient  sequence  between 
areas.  In  addition,  dividing  tasks  into  cards  that  can  be 
separated  allows  for  easier  task  distribution  among 
crew  members,  allowing  mutual  supervision  in  a  crew. 
Features  of  our  job  aid  such  as  allowing  mechanics  to 
sequence  and  distribute  tasks,  the  convenient  size  and 
surface  of  the  cards,  and,  possibly,  increased  ease  of 
reading  the  workcards  (in  compliance  with  the  Patel,  et 
al.’s  (1992)  guidelines)  should  promote  mechanics’  use 
of  the  job  aid,  in  turn  aiding  users  in  recalling  proce¬ 
dures.  Although  our  job  aid  will  not  replace  a  sign-off 
sheet  as  a  quality  control  tool,  it  should  reduce  sign-off 
errors  since  mechanics  no  longer  have  to  rely  on  mem¬ 
ory  to  know  which  tasks  are  complete.  Since  tasks  are 
separated  logically  into  cards,  mechanics  can  check 
cards  as  they  complete  the  tasks. 

Since  our  job  aid  meets  these  objectives,  it  should  re¬ 
duce  errors  associated  with  workcards,  as  the  task 
analysis  predicts.  Omissions  related  to  workcards  not 
matching  mechanics’  individual  work  habits  should  be 
reduced  since  the  job  aid  allows  flexibility  in  the  se¬ 
quence  of  task  areas.  Omissions  related  to  interruptions 
should  also  decrease.  Tasks  are  separated  into  small, 
logical  groups  so  that  a  mechanic  can  quickly  scan  the 
card  he  or  she  was  working  with  before  being  inter¬ 
rupted.  The  workcards’  easier  writing  surface  should 
encourage  mechanics  to  take  notes  about  tasks  inter¬ 
rupted,  tasks  completed,  and  of  discrepancies  found. 
Omissions  and  rule-based  errors  arising  from  mechan¬ 
ics  not  using  the  workcard  should  be  reduced  since  the 
job  aid  was  designed  in  a  way  that  encourages  its  use. 
To  determine  whether  these  predictions  are  valid,  we 
obtained  feedback  from  mechanics  and  observed  them 
using  our  job  aid  while  performing  checks. 


Our  evaluation  of  the  job  aid  consisted  of  the  same 
methodology  we  used  for  task  analysis.  We  ob¬ 


served  mechanics  performing  the  check  using  the  job 
aid,  had  interviews  with  selected  mechanics,  and  dis¬ 
tributed  workcard  evaluations  to  evaluate  and  further 
refine  the  job  aid. 

8.5.1  Direct  Observation 

We  videotaped  a  mechanic  performing  a  lower-level 
2  check  while  using  the  job  aid.  He  rearranged  the 
cards  to  reflect  his  preferred  sequence  for  the  check 
and  followed  the  cards  almost  exactly  during  the 
check.  The  mechanic  frequently  referred  to  the  cards 
to  ensure  he  had  completed  all  tasks  in  sequence.  Af¬ 
ter  he  thought  he  had  completed  the  exterior  checks 
and  referred  to  the  cards,  he  found  that  he  did  not 
check  the  fuel  tank  sump.  In  the  aircraft’s  interior,  the 
mechanic  noted  blown  lights  on  a  piece  of  paper  be¬ 
cause  the  job  aid  he  used  was  a  prototype  made  of 
cardstock  and  not  laminated.  The  mechanic’s  se¬ 
quencing  of  tasks  demonstrated  the  expected  spatial 
sequence;  he  performed  tasks  while  walking  clock¬ 
wise  around  the  aircraft.  General  observation  indi¬ 
cated  that  this  mechanic  followed  our  job  aid’s  task 
sequence  significantly  more  than  the  workcard’ s  task 
sequence. 

8.5.2  First  Workcard  Evaluation 

Appendix  8-C  shows  results  of  a  preliminary  workcard 
evaluation  we  used  for  feedback  after  developing  our 
first  job  aid.  The  placemarker  page  received  a  “useful” 
rating.  This  page  is  a  colored  instruction  card  intended 
be  placed  on  top  of  the  card  stack.  As  a  mechanic 
turned  each  card  over,  the  placemarker  page  separated 
completed  cards  from  those  yet  to  be  performed.  Our 
observations  and  interviews  revealed  that  mechanics 
were  reluctant  to  move  the  placernarker  page  after  they 
completed  tasks  on  a  card.  We  removed  the  place- 
marker  feature  since  it  might  be  more  confusing  than 
helpful.  Mechanics,  instead,  can  use  a  grease  pencil  to 
track  completed  tasks. 

General  results  from  the  first  workcard  evaluation  and 
those  from  subsequent  interviews  with  mechanics  and 
an  inspector  suggested  that  they  found  the  division  of 
tasks  into  small  cards  useful,  that  they  would  rearrange 
the  cards  into  their  own  preferred  order,  and  that  they 
would  find  a  grease  pencil  useful.  In  addition  to  pre¬ 
ferring  the  job  aid  to  the  workcard,  they  indicated  that 
they  would  be  more  likely  to  perform  tasks  in  the  job 
aid’s  order  they  arranged  than  with  the  workcard ’s 
dictated  order.  They  generally  liked  the  card  system 
and  found  it  useful.  Two  suggestions  we  used  to  design 
the  revised  job  aid  were  to  make  the  cards  smaller  and 
to  color-code  cards  by  spatial  areas  of  the  aircraft  so 
that  it  would  be  easier  to  order  the  cards.  Due  to  time 
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constraints,  only  three  mechanics  filled  out  the  prelimi¬ 
nary  workcard  evaluation.  After  revising  job  aid,  we 
distributed  another  workcard  evaluation. 

8.5.3  Second  Workcard  Evaluation 

Seventeen  mechanics  completed  the  second  workcard 
evaluation  after  they  viewed  a  demonstration  of  the  job 
aid.  The  results,  presented  in  Appendix  8-F,  reveal  lit¬ 
tle  difference  between  the  present  workcard  and  the 
proposed  job  aid.  The  only  factor  revealing  a  differ¬ 
ence  between  the  workcard  and  the  job  aid  was  the  me¬ 
chanics’  opinion  that  they  would  perform  the  check  in 
the  order  given.  They  indicated  that  they  seldom  per¬ 
form  tasks  in  the  workcard ’s  order  but  would- 
sometimes  to  usually-perform  tasks  in  the  order  they 
arranged  while  using  the  job  aid.  This  result  is  encour¬ 
aging  given  that  the  job  aid’s  main  goal  is  to  provide  a 
task  order  mechanics  will  follow  so  they  use  the  work- 
card  and  do  not  rely  on  memory.  Mechanics  found 
color  coding  of  cards  (3.65),  division  of  tasks  into  the 
smaller  cards  (3.82),  and  the  grease  pencil  (3.88) 
slightly  less  than  useful  (which  would  be  a  4.0  rating). 
These  findings  are  somewhat  surprising  since  many 
mechanics  make  notes  and  a  mechanic  recommended 
color-coding.  One  mechanic  suggested  that  the  entire 
card  be  color-coded.  Our  question  regarding  the  use¬ 
fulness  of  dividing  tasks  into  smaller  cards  was  proba¬ 
bly  inappropriate  since  tasks  were  divided  so  that 
mechanics  could  arrange  the  sequence  (which  received 
a  favorable  response). 

One  potential  reason  for  the  "neutral  to  slightly  above" 
evaluation  of  the  job  aid  versus  the  workcard  is  that 
many  respondents  did  not  use  the  job  aid  to  perform  a 
check,  but  only  saw  a  demonstration.  Had  they  used  the 
job  aid,  many  mechanics  may  have  been  more  con¬ 
vinced  about  its  usability.  Also,  mechanics  who  had 
been  trained  to  use  workcards  were  reluctant  to  accept 
a  change.  They  seemed  concerned  about  issues  of 
tracking  interim  changes  and  the  ease  of  updating  cards 
for  new  information.  If  lamination  becomes  too  costly, 
there  is  an  alternate  possibility  of  printing  cards  on  card 
stock,  which  is  more  resilient  to  environmental  factors 
than  ordinary  paper.  Such  cards  could  be  used  once  and 
be  updated  as  easily  as  the  workcards.  The  job  aids 
printed  on  card  stock  that  were  used  for  the  DC-9 
lower-level  2  check  we  videotaped  and  reported  in 
8.5.1  appeared  to  work  well. 

Another  possible  reason  for  mechanics’  neutral  re¬ 
sponses  reflects  their  belief  about  the  reliability  of  their 
work.  As  we  previously  discussed,  these  mechanics  are 
experienced  and  extremely  familiar  with  tasks  per¬ 
formed  in  a  check.  They  typically  receive  little,  if  any, 
feedback  about  the  danger  of  interruptions  and  of  fail¬ 


ing  to  use  the  workcard  of  to  follow  its  task  sequence. 
Since  relationships  between  human  error  and  using  the 
workcard  are  not  obvious,  any  possibility  of  increasing 
these  checks’  reliability  is  worth  investigating. 

8.5.4  Overall  Results 

Observations  we  made  of  mechanics  using  the  job  aid 
while  performing  a  check  generally  revealed  closer 
compliance  with  the  task  sequence  the  mechanics  ar¬ 
ranged  while  using  the  job  aid  than  observations  we 
made  of  mechanics  using  traditional  workcards.  Inter¬ 
views  and  informal  discussions  revealed  that  mechanics 
had  generally  favorable  responses  to  the  job  aid.  The 
first  workcard  evaluation’s  results  reflects  this  finding. 
In  contrast,  the  second  workcard  evaluation’s  results 
revealed  mostly  neutral  responses  to  the  job  aid.  Most 
mechanics  completing  the  second  workcard  evaluation 
were  unfamiliar  with  the  goals  of  this  project.  Hence, 
they  were  skeptical  about  the  project  and  logistics  of 
implementing  the  job  aid.  In  contrast,  the  first  workcard 
evaluation  and  direct  observation  involved  a  small 
numbers  of  people  who  understood  the  project’s  goal 
of  increasing  workcard  compliance.  After  other  me¬ 
chanics  begin  using  the  job  aid,  we  expect  initial  neu¬ 
tral  reactions  to  be  followed  by  acceptance  with 
increased  use. 


In  this  study,  we  examined  issues  in  developing  a 
job  aid  for  frequently  performed,  long,  sequential 
tasks  to  increase  reliability  of  task  performance.  Our 
most  important  recommendation  from  this  project  is 
to  design  flexible  job  aids  meet  individual  work 
methods.  To  do  so,  it  is  important  to  identify  factors 
influencing  individual  work  methods.  Our  task 
analysis  found  that  mechanics  performing  low-level 
checks  and  A-checks  use  the  spatial  locations  of  tasks 
and,  sometimes,  perceived  task  difficulty  for  sequenc¬ 
ing  the  tasks.  Other  factors  may  be  more  important 
for  sequencing  less  frequently  performed  checks. 

Separating  tasks  allows  for  a  natural  division  of  work 
and,  more  importantly,  makes  it  easier  for  mechanics 
to  track  completed  tasks.  The  job  aid  should  allow 
mechanics  quickly  to  see  what  tasks  are  completed. 
Further,  sign-offs  for  tasks  located  on  different  air¬ 
craft  sections  should  be  separated  since  generally  they 
are  not  performed  sequentially. 

Another  potential  method  for  helping  mechanics  to 
track  completed  tasks  is  a  bar  code  reader.  A  bar  code 
could  be  printed  on  each  card  of  a  check.  After  a  me¬ 
chanic  completes  all  tasks  on  a  card,  he  or  she  could 
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scan  the  bar  code,  using  a  small,  lightweight  com¬ 
puter  attached  to  his  or  her  belt.  After  the  check  is 
complete,  the  computer  could  identify  any  tasks  me¬ 
chanics  missed.  After  mechanics  are  sure  that  all  tasks 
are  completed,  they  can  do  their  "sign-offs"  either 
manually  or  with  the  computer  (when  computer  rec¬ 
ognition  of  signatures  becomes  common).  Either  ap¬ 
proach  would  significantly  reduce  mechanics'  current 
reliance  on  memory.  As  bar  code  readers  are  rela¬ 
tively  inexpensive,  airlines  should  further  investigate 
this  option. 

The  job  aid  must  be  resilient  to  environmental  factors 
and  compatible  with  task  factors.  Task  analysis 
should  identify  conditions  under  which  mechanics 
will  use  the  job  aid.  The  job  aid  must  not  physically 
hinder  users  performing  their  tasks. 

Mechanics  must  understand  the  importance  of  using 
workcards,  especially  the  ways  interruptions  and  dis¬ 
tractions  can  lead  mechanics  to  omit  tasks.  Factors 
such  as  weather,  absences  by  co-workers,  reassign¬ 
ment,  and  time  pressure  all  contribute  to  the  potential 
for  distractions. 

Finally,  workcards,  as  a  form  of  checklists,  must  meet 
objectives  of  checklists  (Degani  and  Wiener,  1990; 
1993).  Workcards  should  aid  users  to  recall  proce¬ 
dures  by  outlining  a  convenient  sequence  for  motor 
movements  and  eye  fixations.  Workcards  should 
permit  mutual  supervision  within  a  crew,  as  well  as 
helping  a  crew  distribute  tasks  among  themselves. 
Taken  together,  these  factors  should  increase  a  work- 
card’s  ability  to  function  as  a  control  tool  for  man¬ 
agement  and  government  regulators,  thereby 
increasing  the  checks*  reliability. 
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Results  of  Present  Workcard  Evaluations 


1.  Summary  Statistics 

Number  of  respondents  =  8 

Age  of  respondents;  Mean=40.38  sd=7.73 

Years  worked  as  a  mechanic;  Mean=17.4  sd=9.80 

Average  number  of  lower-level  checks  performed  per  month; 
Mean=14.25  sd=8.25 


n.  Open-Ended  Questions 

1 .  Do  you  normally  perform  the  tasks  on  a  lower-level  2  check  in  the  same  order  every  time  you 
do  the  check? 

Yes;  3 

No;  4 

Depending  on  aircraft  type;  1 

2.  Normally,  how  do  you  sequence  the  tasks  you  must  perform  to  complete  a  lower-level  2 
check? 

Starting  at  the  nose  of  aircraft,  I  wrap  around  wings  and  empennage  finishing 
at  the  nose  again. 

Nose  to  left  side  of  aircraft  to  nose. 

Sometime  start  on  the  outside,  sometimes  start  inside. 

Start  at  nose,  work  way  around. 

Outside,  inside,  work  release  items. 

Inside  right  to  left,  inside  back  to  front. 

Outside,  inside,  pilot  items. 

Habit. 

3.  If  you  are  doing  the  check  with  another  person,  how  does  this  change  your  strategy  for 
performing  the  check? 

Subject  1;  Assistant  on  check  would  service  tires,  APU  oil,  engine  oil  and  CSD  oil  and 
hydraulic  fluid. 

Subject  2;  None. 

Subject  3;  One  person  will  do  the  outside,  the  other  one  will  do  the  inside. 


144 


Chapter  8 

Appendix  -  Improving  the  Reliability  of  Maintenance  Checklists 

Subject  4: 

Usually  split  inside  and  outside. 

Subject  5: 

Depends  on  level  of  experience. 

Subject  6: 

None. 

Subject  7: 

None. 

Subject  8: 

One  man  assigned  to  inside.  One  man  outside. 

4. 

What  do  you  do  when  you  find  a  discrepancy,  e.g.,  do  you  make  a  note  to  fix  it  after  you  are 
finished  with  the  check,  or  do  you  fix  it  as  soon  as  you  find  it? 

Subject  1: 

Make  notes. 

Subject  2: 

Make  note  of  discrepancy. 

Subject  3: 

Made  a  note  and  fix  it  after  the  check  is  done. 

Subject  4: 

Make  a  note  usually  unless  able  to  fix  on  spot. 

Subject  5: 

Fix  after. 

Subject  6: 

Make  a  note. 

Subject?: 

Fix  after  the  check. 

Subject  8: 

Make  a  note. 

5. 

Could  you  please  comment  on  the  usefulness  of  the  workcard,  e.g.,  do  you  need  to  refer  to  the 
workcard  while  performing  the  check? 

Subject  1: 

No,  unless  there  is  a  new  revision. 

Subject  2: 

No. 

Subject  3: 

Sometimes. 

Subject  4: 

Used  as  guide  since  things  checked  are  usually  more  than  required. 

Subject  5: 

No. 

Subject  6: 

No. 

Subject  7: 

Sometimes. 

Subject  8: 

The  first  4  to  5  times  you  do  the  check  on  any  specific  a/c  after  that  no. 

ni.  General  Questions  on  the  Usefulness  of  the  Present  workcards 

1 .  How  useful  do  you  find  the  workcard? 

Mean=4  sd=0.535 

[0=  of  no  use  2=  not  very  useful  4=  useful 
6=  considerably  useful  8=  extremely  useful] 

2.  How  often  do  you  refer  to  the  workcard? 

Mean=4.125  sd=1.727 

[0=  always  2=  usually  4=  sometimes  6=  seldom  8=  never] 

3.  Would  you  prefer  a  workcard  that  is: 

Mean=4.688  sd=1.945 

[0=  more  concise  4=  about  the  same  8=  more  detailed 
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4.  How  would  you  rate  the  ease  of  understanding  of  the  workcard? 

Mean=5.125  sd=1.959 

[0=  very  difficult  4=  moderately  easy  8=  very  easy] 

5.  Do  you  have  any  problems  handling  the  workcard? 

Mean=6.625  sd=1.408 
[0=  always  4=  sometimes  8=  never] 

6.  Do  you  perform  the  tasks  in  the  order  given  by  the  workcard? 

Mean=2.750  sd=1.389 
[0=  never  4=  sometimes  8=  always] 

7.  When  do  you  sign  off  complete  items  on  the  workcard? 

Five  mechanics  responded  at  end  of  workcard. 

One  mechanic  responded  between  intermittently  and  end  of  workcard. 
One  responded  after  every  section. 

One  responded  after  every  task. 
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APPENDIX  8-B 

Mechanics*  Ratings  of  Probability  of  Discrepancy  and 
Difficulty  of  B-737  Lower-Level  2  Check  Tasks 

TTie  approximate  likelihood  of  finding  a  discrepancy  was  rated: 

0=  never  4=  sometimes  8=  always 


The  difficulty  of  performing  the  task  was  rated: 

0=  very  easy  4=  moderately  easy  8=  very  difficult 


Task 

Prob.  of 

Discrepancy: 
mean  (sd) 

Difficulty  of 

Task:  mean  (sd) 

Check  left  engine  inlet  and  reverser  area. 

1.2  (2.0) 

2.2  (2.4) 

Check  right  engine  inlet  and  reverser  area. 

2.5  (1.3) 

1.6  (1.6) 

Check  brakes  for  wear  with  pressure  applied. 

2.9  (1 .7) 

4.0  (2.3) 

Check  main  landing  tires  for  serviceability. 

3.5  (1.3) 

2.6  (1.4) 

Check  nose  landing  tires  for  serviceability. 

2.8  (1.5) 

1.8  (1.1) 

Check  nose  tire  pressure. 

4.2  (1.8) 

3.3  (2.8) 

Check  main  landing  tire  pressure. 

3.7  (2.0) 

1.8  (1.6) 

Accomplish  a  visual  check  of  MLG  wheels  for 
broken  or  missing  tie  bolts. 

2.2  (2.2) 

2.9  (2.1) 

Clean  MLG  strut  piston  with  solvent.  Clean  MLG 
downlock  viewers/indicators. 

3.3  (1.8) 

1.6  (1.6) 

Clean  NLG  strut  piston  with  solvent.  Clean  NLG 
downlock  viewers/indicators. 

3.3  (1.2) 

2.3  (1.6) 

Check  fuselage  for  obvious  damage  as  viewed 
from  the  ground. 

3.1  (2.0) 

1.8  (2.2) 

Check  empennage  for  obvious  damage  as 
viewed  from  the  ground. 

2.6  (1.1) 

2.6  (2.4) 

Check  wings  for  obvious  damage  as  viewed 
from  the  ground. 

2.3  (1.1) 

1.7  (1.4) 

Check  tail-skid  (737-400  only) 

1.8  (2.5) 

0.5  (0.6) 

Check  engine  fire  bottle  pressure. 

1.1  (0.9) 

0.9  (0.7) 

Check  APU  fire  bottle  disc  and  thermal  relief 
indicator. 

0.6  (0.7) 

1.1  (2.4) 
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Task 

Prob.  of 
Discrepancy: 
mean  (sd) 

Difficulty  of 

Task:  mean  (sd) 

indicator. 

Check  exterior  lights  for  proper  operation. 

4.1  (0.9) 

1.9  (1.8) 

Check  fuel  tank  sumps. 

3.3  (1.2) 

2.1  (1.1) 

Service  hydraulic  fluid  for  standby  system. 

3.3  (1.4) 

2.5  (1.4) 

Service  hydraulic  fluid  for  system  B. 

2.7  (1.7) 

1.9  (1.8) 

Service  hydraulic  fluid  for  system  A. 

2.7  (1.5) 

1.7  (1.6) 

Service  auxiliary  power  unit  oil  to  NON  RON 
aircraft. 

3.9  (1.8) 

2.7  (1.9) 

Service  engine  oil  for  engine  #1 . 

4.9  (2.3) 

1.1  (1.4) 

Service  engine  oil  for  engine  #2. 

4.4  (2.6) 

0.9  (0.6) 

Service  constant  speed  drive  engine  #1 . 

3.2  (1.6) 

1.6  (1.4) 

Service  constant  speed  drive  engine  #2. 

2.6  (1 .4) 

0.8  (0.8) 

Service  oxygen-crew,  portable. 

2.4  (1.4) 

2.4  (2.0) 

Check  attendants'  seats  for  proper  operation 
and  condition. 

2.3  (1.0) 

2.4  (1.9) 

Ensure  outboard  seat  in  the  emergency  exit  row 
has  a  non-standard  thinner  seat  bottom  cushion 
installed. 

2.5  (1.9) 

0.6  (0.6) 

Check  that  a  yellow  lifevest  is  installed  under 
each  seat. 

5.7  (2.3) 

1.9  (1.8) 

Check  LH  overhead  stowage  bin  row  (10)  for  8 
spare  yellow  passenger  life  vests. 

3.0  (2.1) 

1.2  (1.4) 

Check  forward  LH  closet  for  2  each  yellow 
demo  lifevests. 

2.4  (1.8) 

0.6  (0.6) 

Check  LH  emergency  equipment  bin  for  2  demo 
lifevests. 

3.0  (2.3) 

1.1  (1.7) 

Check  protective  breathing  equipment  for 
serviceability. 

0.8  (1.0) 

1.2  (1.4) 

Check  lavatory  flush  pumps/timers. 

2.7  (2.3) 

1.8  (1.8) 

Check  emergency  lighting  system. 

2.7  (1.9) 

2.8  (1.9) 
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Task 

Prob.  of 
Discrepancy: 
mean  (sd) 

Difficulty  of 

Task:  mean  (sd) 

Check  and  repair  the  entrance  area  for 
appearance  and  condition. 

3.3  (1.2) 

Check  cabin  area  for  appearance  and  condition. 

4.3  (1 .8) 

Check  galley  area  for  general  appearance  and 
condition. 

2.4  (1 .8) 

2.2  (1.6) 

Check  forward  lavatories  for  general 
appearance  and  condition. 

2.4  (1 .5) 

2.0  (1.2) 

Check  rear  lavatories  for  general  appearance 
and  condition. 

2.8  (1 .3) 

1.9  (1.4) 
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APPENDIX  8-C 

Sequence  of  Tasks  for  Lower  Level-Check  2  on  B-737 


Five  mechanics  completed  this  evaluation.  The  first  two  respondents  were  also  videotaped 
performing  this  check. 


In  Table  A2,  the  order  in  which  each  task  was  performed  is  indicated  by  its  task  number.  Mechanics 
ml-m5  completed  the  evaluation  and  are  denoted  by  mlq-m5q.  Mechanics  ml  and  m2  were  also 
videotaped  and  are  denoted  by  ml-v  and  m2-v.  Note  that  mechanic  2  split  the  check  with  another 
mechanic,  so  many  tasks  were  not  observed.  For  mechanic  ml,  some  tasks  could  not  be  seen  due  to 
the  video  camera’s  position. 
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Table  Al.  Workcard  Order  for  Tasks  1-27 


Task# 

Description 

1 

Check  engines  inlet  and  reverser  area. 

2 

Check  brakes  for  wear  with  pressure  applied. 

3 

Check  tires  for  serviceability. 

4 

Check  tire  pressure. 

5 

Accomplish  a  visual  check  of  the  main  landing  gear  for  broken  or  missing  tie 
bolts. 

6 

Clean  MLG  &  NLG  strut  piston  with  solvent.  Clean  MLG  &  NLG  downlock 
viewers/indicators. 

Check  fuselage,  empennage,  and  wings  for  obvious  damage  or  irregularities 

7 

as  viewed  from  the  ground. 

8 

Check  tail  skid. 

9 

Check  engine  fire  bottle  pressure. 

10 

Check  APU  fire  bottle  discharge  disc  (yellow)  and  thermal  relief  disc  (red). 

11 

Check  exterior  lights  for  proper  operation. 

12 

Fuel  tank  sumps. 

13 

Hydraulic  fluid  (System  A,  B,  and  Standby). 

14 

Auxiliary  Power  Unit  Oil. 

15 

Engine  oil. 

16 

Constant  speed  drive  #1 ,  #2. 

17 

Oxygen-Crew,  portable. 

18 

Attendants'  seats  for  proper  operation  and  condition. 

19 

Ensure  outboard  seat  in  the  emergency  exit  row  has  a  non-standard  thinner 
seat  bottom  cushion  installed. 

20 

Check  passenger  life  vest,  for  aircraft  that  are  equipped  for  over  water 
operation. 

21 

Protective  breathing  equipment  (PBE)  for  serviceability. 

22 

Lavatory  flush  pumps/timers. 

23 

Emergency  lighting  system. 

24 

Entrance  area  for  appearance  and  condition. 

25 

Galley  area  for  general  appearance  and  condition. 

26 

Cabin  area  for  general  appearance  and  condition. 

27 

Lavatories  for  general  appearance  and  condition. 

V 


151 


152 


Appendix  ~  Improving  the  Reliability  of  Maintenance  Checklists 


Chapter  8 


Order 

m-q 

ml-v 

m2-q 

m2V 

mS-q 

m4"q 

m5-q 

46 

19 

14* 

47 

18 

15* 

20 

18* 

49 

24 

19* 

50 

21 

23* 

51 

20 

25* 

52 

26 

26* 

53 

8 

54 

11* 

55 

12* 

56 

15* 

57 

25* 

*  Asterisks  represent  tasks  performed  by  other  mechanics  or  not  observed  dne  to  video 
restrictions^ 
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APPENDIX  8-D 
Tasks  Occurring  Sequentially 

Tasks  which  follow  each  heading  task  are  listed. 

Check  left  engine  inlet  and  reverser  area. 

Check  main  landing  tire  pressure. 

Accomplish  a  visual  check  of  MLG  wheels  for  broken  or  missing  tie  bolts. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Service  constant  speed  drive  engine  #1. 

Service  constant  speed  drive  engine  #2. 

Check  right  engine  inlet  and  reverser  area. 

Check  main  landing  tires  for  serviceability. 

Check  main  landing  tire  pressure. 

Clean  MLG  stmt  piston  with  solvent.  Clean  MLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Check  fuel  tank  sumps. 

Check  brakes  for  wear  with  pressure  applied. 

Check  main  landing  tire  pressure. 

Accomplish  a  visual  check  of  MLG  wheels  for  broken  or  missing  tie  bolts. 
Check  empennage  for  obvious  damage  as  viewed  from  the  ground. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Check  main  landing  tires  for  serviceability. 

Check  main  landing  tire  pressure. 

Accomplish  a  visual  check  of  MLG  wheels  for  broken  or  missing  tie  bolts. 
Clean  MLG  stmt  piston  with  solvent.  Clean  MLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Service  constant  speed  drive  engine  #1. 
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Check  nose  landing  tires  for  serviceability. 

Check  nose  tire  pressure. 

Clean  NLG  strut  piston  with  solvent.  Clean  NLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  nos®  tire  pressure. 

Clean  NLG  strut  piston  with  solvent.  Clean  NLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  main  landing  tire  pressure. 

Accomplish  a  visual  check  of  MLG  wheels  for  broken  or  missing  tie  bolts. 
Clean  MLG  strut  piston  with  solvent.  Clean  MLG  downlock  viewers/indicators. 

Accomplish  a  visual  check  of  MLG  wheels  for  broken  or  missing  tie  bolts. 

Clean  MLG  strut  piston  with  solvent.  Clean  MLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Clean  MLG  strut  piston  with  solvent.  Clean  IVILG  downlock  viewers/indicators. 
Check  empennage  for  obvious  damage  as  viewed  from  the  ground. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Service  constant  speed  drive  engine  #2. 

Clean  NLG  strut  piston  with  solvent.  Clean  NLG  downlock  viewers/indicators. 
Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  exterior  lights  for  proper  operation. 

Check  fuselage  for  obvious  damage  as  viewed  from  the  ground. 

Check  empennage  for  obvious  damage  as  viewed  from  the  ground 
Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Check  exterior  lights  for  proper  operation. 

Check  fuel  tank  sumps. 

Service  APU  unit  oil  to  NON  RON  aircraft. 

Service  constant  speed  drive  engine  #1 . 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 
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Check  empennage  for  obvious  damage  as  viewed  from  the  ground. 

Service  APU  unit  oil  to  NON  RON  aircraft. 

Check  wings  for  obvious  damage  as  viewed  from  the  ground. 

Check  fuel  tank  sumps. 

Service  hydraulic  fluid  for  standby  system. 

Service  constant  speed  drive  engine  #1 . 

Service  constant  speed  drive  engine  #  2. 

Service  oxygen-crew,  portable. 

Service  hydraulic  fluid  for  standby  system. 

Service  hydraulic  fluid  for  system  B. 

Service  hydraulic  fluid  for  system  B. 

Service  hydraulic  fluid  for  system  A. 

Service  oxygen-crew  portable. 

Check  LH  emergency  equipment  bin  for  2  demo  lifevests. 

Check  protective  breathing  equipment  for  serviceability. 

Check  lavatory  flush  pumps/timers. 

Check  forward  lavatories  for  general  appearance  and  condition. 

Check  attendants'  seats  for  proper  operation  and  condition. 

Check  that  a  yellow  lifevest  is  installed  under  each  seat. 

Check  LH  emergency  equipment  bin  for  2  demo  lifevests. 

Check  lavatory  flush  pumps/timers. 

Check  emergency  lighting  system. 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 

Check  forward  lavatories  for  general  appearance  and  condition. 

Ensure  outboard  seat  in  the  emergency  exit  row  has  a  non-standard  thinner  seat  bottom 
cushion  installed. 

Check  that  a  yellow  lifevest  is  installed  under  each  seat. 

Check  emergency  lighting  system. 
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Check  cabin  area  for  appearance  and  condition. 

Check  that  a  yellow  lifevest  is  installed  under  each  seat. 

Check  LH  overhead  stowage  bin  row  (10)  for  8  spare  yellow  passenger  life  vests. 
Check  LH  emergency  equipment  bin  for  2  demo  lifevests. 

Check  protective  breathing  equipment  for  serviceability. 

Check  lavatory  flush  pumps/timers. 

Check  cabin  area  for  appearance  and  condition. 

Check  LH  overhead  stowage  bin  row  (10)  for  8  spare  yellow  passenger  life  vests. 
Check  cabin  area  for  appearance  and  condition. 

Check  forward  LH  closet  for  2  each  yellow  demo  lifevests. 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 

Check  cabin  area  for  appearance  and  condition. 

Check  LH  emergency  equipment  bin  for  2  demo  lifevests. 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 

Check  cabin  area  for  appearance  and  condition. 

Check  protective  breathing  equipment  for  serviceability. 

Check  emergency  lighting  system. 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 

Check  cabin  area  for  appearance  and  condition. 

Check  lavatory  flush  pumps/timers. 

Check  rear  lavatories  for  general  appearance  and  condition. 

Check  emergency  lighting  system. 

Check  forward  lavatories  for  general  appearance  and  condition. 

Check  and  repair  the  entrance  area  for  appearance  and  condition. 

Check  forward  lavatories  for  general  appearance  and  condition. 

Check  cabin  area  for  appearance  and  condition. 

Check  rear  lavatories  for  general  appearance  and  condition. 
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APPENDIX  8-E 

First  Evaluation  Feedback  on  the  Proposed  Job  Aid 

I.  Mechanics'  Ratings  of  Job  Aid 

Three  mechanics  (Ml -M3)  responded. 


Question 

M1 

M2 

M3 

Mean 

How  useful  would  you  find  the  placemarker  page? 

0=of  no  use  4=useful  8=extremely  useful 

5 

3 

4 

How  useful  do  you  think  the  division  of  tasks  into 
small  cards  would  be? 

0=of  no  use  4=useful  8=extremely  useful 

5 

6 

5 

5.3 

Would  you  rearrange  the  cards  to  suit  your  individual 
work  habits? 

0=never  4=sometimes  8=always 

7 

8 

5 

6.7 

Would  you  read  the  interim  page  at  the  end  of  the 
"official"  w/c  before  starting  the  check? 

0=never  4=sometimes  8=always 

4 

3 

3.5 

Would  you  use  the  grease  pencil  to  make  notes  while 
completing  the  check? 

0=never  4=sometimes  8=always 

7 

8 

1 

5.3 

How  would  you  rate  the  size  of  the  cards? 

0=too  small  4=about  right  8=too  big 

5 

6 

1 

4 

How  useful  do  you  find  the  present  w/c  system? 

0=of  no  use  4=useful  8=extremely  useful 

4 

1 

5 

3.3 

How  useful  do  you  think  the  proposed  job  aid  would 
be? 

0=of  no  use  4=useful  8=extremely  useful 

6 

6 

5 

5.7 

Do  you  perform  the  tasks  in  the  order  given  by  the 
present  w/c? 

0=never  4=sometimes  8=always 

0 

0 

Would  you  perform  the  tasks  in  the  order  you 
arranged  using  the  job  aid? 

0=never  4=sometimes  8=always 

6 

8 

5 

6.3 

159 


Appendix  -  Improving  the  Reliability  of  Maintenance  Checklists 


Chapter  8 


Question 

M1 

M2 

M3 

Mean 

How  often  do  you  refer  to  the  present  workcard  as 
you  perform  a  lower-level  2  check? 

0=never  4=sometimes  8=always 

4 

0 

5 

3 

How  often  would  you  refer  to  the  job  aid  as  you 
perform  a  lower-level  2  check? 

0=never  4=sometimes  8=always 

6 

6 

3 

5 

How  often  do  you  refer  to  the  present  workcard  as 
you  perform  an  A-check? 

0=never  4=sometimes  8=always 

5 

5 

7 

5.7 

How  often  would  you  refer  to  the  job  aid  as  you 
perform  an  A-check? 

0=never  4=sometimes  8=always 

6 

6 

7 

6.3 

n.  Open-Ended  Questions 

1 .  Comments  and  suggestions  on  the  design  of  the  cards: 

a.  Size  of  the  cards 

Subject  1:  Could  be  a  little  smaller  to  stow  in  pockets  when  both  hands  are 

needed. 

Subject  2:  Shirt  pocket  with  a  grommet  to  allow  the  cards  to  fan  open,  or  some 

firm  type  of  clip. 

Subject  3:  Good  size  for  information  that  is  on  each  card. 

b.  Groupings  of  the  tasks 

Subject  1:  OK-after  rearranging  to  preference. 

Subject  2:  From  aircraft  access  (fwd  med)  toward  nose  and  around  to  right  buy 

areas  (normal  course). 

Subject  3:  Good  idea.  I  think  it's  easier  to  start  at  the  nose  gear  and  continue 

around  the  aircraft  in  one  complete  circle. 

c.  Placemarker/instmctions  page 

Subject  1 :  OK. 
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Instructions  on  front  as  a  cover.  Check  boxes  at  item  number  with 
back  page  having  colored  stripes-"Check  off  area"  to  recall  page  with 
check. 

Once  I  got  used  to  doing  a  check  on  an  aircraft,  I  don't  think  I  would 
use  the  placemarker/instruction  card  and  just  use  the  sign-off  sheet. 

d.  Wording  of  the  cards/instructions 
Wouldn’t  hurt  to  go  into  more  detail. 

Revision  date  in  large  print  to  match  sign-off  sheet  date.  Common 
abbreviation  naming  component  only.  Include  limits.  Leave  out 
procedure  (manuals  dictate  procedure). 

Simplified  and  easy  to  understand. 

e.  Ease  of  understanding  the  instmctions 
Good. 

Very  brief-reference  changes  only-new  or  limited  experience 
personnel  should  consult  M/M  until  they  are  confident  in  their 
procedure. 

The  cards  are  very  easy  to  understand. 

f.  Ease  of  rearranging  the  order  of  the  cards 

Subject  1:  OK. 

Subject  2:  Not  necessary  if  color-code  by  geographic  areas  of  aircraft. 

Subject  3:  Rearrange  the  cards  in  order  of  doing  the  check. 

2.  How  well  do  you  think  this  idea  can  be  extended  to  other  ehecks? 

Subject  1:  The  more  involved  the  check,  the  more  useful  the  cards. 

Subject  2:  Very  well. 

Subject  3:  Very  easily. 
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3.  General  comments 

Subject  1:  I  like  the  card  system  better. 

Subject  2:  Its  nice  to  see  that  people  are  interested  in  approaching  these  tasks  in  a  real-  • 

world  manner. 
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Appendix  8-F 

Evaluation  Feedback  on  Revised  Job  Aid 

Statistical  Data  on  Respondents 
N=17 

Age  =  36.47(8.15)  years 

Number  of  years  in  civil  aviation  =  14.35(7.58) 

Number  of  years  as  a  mechanic  =  12.94(6.95) 

Number  of  years  as  an  inspector  =  0.29(0.99) 

Number  of  years  performing  lower-level  2  checks  =  9.59(6.76) 

Approximate  number  of  lower  level  2  checks  performed  in  a  month  =  15.85(9.07) 
Number  of  years  performing  A-checks  =  9.59(6.76) 

Approximate  number  of  A-checks  performed  in  a  month  =  4.65(4.00) 
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Question 

Present 

Workcard 

Mean  (sd) 

Job  Aid 

Mean  (sd) 

How  would  you  rate  the  ease  of  readability  of  the 
text? 

0=terrible  2=poor  4=fair  6=good  8=excellent 

5.47(1.42) 

6.12(1.27) 

In  general,  how  easy  is  the  information  to 
understand? 

0=very  difficult  4=moderately  easy  8=very  easy 

6.06(2.19) 

6.12(1.65) 

How  would  you  rate  the  effort  required  in 
locating  a  particular  task? 

0=very  difficult  4=moderately  easy  8=very  easy 

5.35(2.42) 

5.59(1.77) 

What  would  be  the  chance  of  you  missing  a  sign- 
off  or  a  task? 

0=always  2=usually  4=sometimes  6=seldom 
8=never 

5.94(1.84) 

6.35(1.27) 

How  would  you  rate  the  ease  of  physically  using 
the  workcard/job  aid? 

0=very  difficult  4=moderately  easy  8=very  easy 

5.47(2.10) 

6.06(1.92) 

i _ 

Would  you  perform  the  tasks  in  the  order  given 
by  the  workcard/job  aid? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

2.47(2.40) 

5.18(2.40) 

How  often  do/would  you  refer  to  the 
workcard/job  aid  as  you  perform  a  lower-level  2 
check? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

5.12(2.42) 

5.41(2.09) 

How  often  do/would  you  refer  to  the 
workcard/job  aid  as  you  perform  an  A-check? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

6.18(1.85) 

6.47(1.59) 

How  useful  do  you  find  the  workcard/job  aid? 
0=of  no  use  4=useful  8=extremely  useful 

4.06(2.19) 

5.12(2.12) 

How  useful  would  you  find  the  color-coding  of 
the  tasks  into  areas? 

0=of  no  use  4=useful  8=extremely  useful 

3.65(1.90) 
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How  useful  do  you  think  the  division  of  tasks 
into  small  cards  would  be? 

0=of  no  use  4=useful  8=extremely  useful 

3.82(1.98) 

Would  you  rearrange  the  cards  to  suit  your 
individual  work  habits? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

5.76(2.44) 

Would  you  read  the  interim  page  at  the  end  of  the 
workcard  before  starting  the  check? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

6.00(2.21) 

Would  you  use  the  grease  pencil  to  make  notes 
while  completing  the  check? 

0=never  2=seldom  4=sometimes  6=usually 
8=always 

3.88(2.34) 
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This  project’s  objective  is  to  provide  human  factors 
inspection  expertise  to  support  the  Visual 
Inspection  Research  Program  (VIRP).  Note:  The 
material  in  this  chapter  is  the  result  of  a  collaborative 
effort  among  many  organizations  and  is  not  solely  the 
work  of  C.  G.  Drury,  SUNY  at  Buffalo,  or  of  Galaxy 
Scientific  Corporation. 


Over  the  past  two  decades  there  have  been  several 
studies  of  human  reliability  in  aircraft  structural 
inspection  (Rummel,  Hardy,  &  Cooper,  1989;  Spencer 
&  Schurman,  1994;  and  Murgatroyd,  Worrall,  & 
Waites,  1994).  All  of  these  studies  to  date  have 
examined  the  reliability  of  Non-Destructive  Inspection 
(NDI)  techniques,  such  as  eddy-current  or  ultrasonic 
technologies.  However,  over  80%  of  civil  aircraft 
inspection  does  not  use  NDI  and  is  classified  as  Visual 
Inspection  (Goranson  &  Rogers,  1983).  Both  the  FAA 
(National  Aging  Aircraft  Research  Program  Plan, 
1993,  p.  26,  p.  35)  and  the  ATA  have  recognized  the 
need  for  equivalent  studies  of  the  reliability  of  visual 
inspection  as  a  research  priority. 

Flight  safety  is  dependent  upon  airframe  integrity;  for 
the  civil  airline  fleet,  this  includes  the  detection  and 
repair  of  structural  defects  as  they  appear.  Data  on 
airframe  structural  forces,  material  characteristics,  and 
models  of  crack  growth  are  used  in  the  Maintenance 
Steering  Group-3  (MSG-3)  process  to  determine  safe 
inspection  schedules.  This  assumes  that  there  are 
multiple  inspection  opportunities  between  the  time  a 
crack  becomes  detectable  and  the  time  it  compromises 
safety.  This  process  is,  thus,  very  sensitive  to 
assumptions  about  crack  detectability.  For  example, 
overestimation  of  inspection  reliability  would  lead  to 
longer  inspection  intervals,  compromising  safety. 
Conversely,  underestimation  of  inspection  reliability 
would  lead  to  shorter  intervals,  increasing  costs 
because  of  unnecessary  inspection. 


While  there  is  a  need  to  obtain  accurate  measures  of  in- 
service  visual  inspection  reliability,  there  is  also  a 
parallel  need  to  understand  the  process  of  aircraft 
visual  inspection  to  improve  it.  There  is  a  large  body  of 
literature  on  visual  inspection  in  the  manufacturing 
industry  (e.g.,  Drury,  1992),  and  an  increasing  number 
of  papers  applying  this  to  aircraft  inspection  (e.g., 
Drury,  1995).  However,  there  are  still  no  on-aircraft 
studies  which  quantify  the  effects  of  the  many 
variables  affecting  human  factors  in  visual  inspection. 
Thus,  a  second  major  goal  of  the  VIRP  is  to  provide 
quantitative  evaluations  of  the  effectiveness  of  visual 
inspection  enhancements. 


Quantifying  visual  inspection  is  inherently  more 
complex  than  quantifying  NDI.  Visual  inspection 
uses  many  senses  and  is  expected  to  detect  many 
indications  beyond  cracks.  It  may  be  applied  to  many 
different  structures  and  surface  treatments. 

Bobo  and  Puckett  ( 1994),  in  the  FAA's  latest  Advisory 
Circular  on  Visual  Inspection  for  Aircraft,  use  the 
following  definition: 

Visual  Inspection  is  the  process  of  using  the 
eye,  alone  or  in  conjunction  with  various 
aids,  as  the  sensing  mechanism  from  which 
judgments  may  be  made  about  the  condition 
of  a  unit  to  be  inspected. 

Visual  inspection  involves  using  the  "eye,  alone  or 
with  various  aids,"  and  also  shaking,  listening,  feeling, 
and  sometimes  smelling,  the  aircraft  and  its 
components.  Additionally,  the  process  of  any 
inspection  can  be  analyzed  as  a  combination  of  various 
functions,  the  two  most  important  functions  are  search 
and  decision-making  (e.g.,  Latorella  &  Drury,  1992). 
In  visual  inspection,  a  search  process  uses  most  of  the 
human  body's  senses  to  detect  and  locate  an  indication. 
There  is  then  a  secondary  process  of  combining 
relevant  knowledge,  sensory  input,  and  pertinent  logic 
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to  determine  if  the  indication  represents  a  flaw.  The 
inspector  must  then  make  a  decision  whether  or  not 
this  flaw  is  sufficiently  sensitive  to  pose  a  risk  to  the 
continued  safe  operation  of  the  aircraft  or  aircraft  part. 

The  Visual  Inspection  Research  Program  uses  the 
following  definition  of  ’’Visual  Inspection": 

Visual  inspection  is  the  process  of 
examination  and  evaluation  of  systems  and 
components  by  use  of  human  sensory^  systems, 
aided  only  by  mechanical  enhancements  to 
sensofy  input,  such  as  magnifiers,  dental 
picks,  stethoscopes,  and  the  like.  The  visual 
input  to  the  inspection  process  may  be 
accompanied  by  such  behaviors  as  listening, 
feeling,  smelling,  shaking,  twisting,  etc. 

In  addition  to  defining  the  process  of  visual  inspection, 
definitions  of  both  the  types  of  indications,  i.e., 
potential  defects  detectable  with  visual  inspection  and 
the  structure  on  which  this  inspection  is  practiced,  need 
to  be  addressed. 

The  types  of  indication  possible  in  aircraft  structures 
were  derived  from  findings  at  The  Aging  Aircraft  Non- 
Destructive  Inspection  Center  (AANC)  and  on  other 


documents  relating  to  inspection.  A  two-level 
classification  scheme  was  developed;  each  major 
heading  was  given  a  two-digit  number  ending  in  zero. 
Below  this  level,  individual  indication  types  shared  the 
same  first  digit  with  the  appropriate  major  heading. 
Table  9. 1  shows  the  current  version  of  this  scheme, 
which  can  be  expanded  or  modified  as  needed. 

To  fully  characterize  an  indication  on  an  aircraft,  it  is 
necessary  to  know  the  type  of  indication  (Table  9. 1 ) 
and  the  structure  on  which  it  is  found.  As  results  of  the 
baseline  inspection  of  the  fuselage  area  of  the  AANC's 
Boeing-737  test  bed  became  available,  the  findings 
were  classified  into  the  two-level  scheme  shown  in 
Table  9.2,  next  page  .  This  table  only  includes 
structural  items  needed  in  the  current  research;  there 
are  obviously  many  more  structural  elements  on  an 
aircraft.  As  with  Table  9. 1 ,  this  classification  scheme 
gives  sufficient  detail  for  the  test  bed  used  in  VIRP,  but 
should  be  expanded  and  modified  as  necessary  to 
better  characterize  visual  inspection  tasks. 

From  the  definitions  given  in  this  section,  the  VIRP 
was  able  to  design  representative  experimental 
evaluations. 


CODE 

DESCR8PTOR 

20 

Wear  and  Tear 

21 

Loose 

22 

Pulled 

23 

Bent 

24 

Dent 

25 

Scratch 

26 

Frayed 

27 

Leaking 

28 

Lighting  Hole 

30 

Corrosion 

31 

Pillowing 

32 

Exfoliation 

33 

Intergranular 

34 

Material  Missing 

40 

Broken 

41 

Crack 

42 

Disbond 

43 

Delamination 

44 

Part  Missing 
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CODE 

DESCRIPTOR 

10 

Skin 

11 

Doubler 

12 

Extension  Skin 

13 

Interior  Skin 

14 

Bulkhead 

15 

Panel 

20 

Fasteners 

21 

Rivet 

22 

Screw 

23 

Bolt 

30 

Support  Structure 

31 

Frame 

32 

Stringer 

33 

Track 

34 

Bracket 

35 

Web 

36 

Mount 

37 

Clip 

40 

Other  Structure 

41 

Rod 

42 

Strap 

50 

Other  Material 

51 

Seal 

52 

Paint 
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EXPERIMENTS  . 


The  research  team  responsible  for  designing, 

conducting,  and  analyzing  the  VTRP  experiments 
includes  personnel  from  Sandia  National 
Laboratories/AANC,  SAIC,  AEA  (U.K,)  as  well  as 
State  University  of  New  York  (SUNY)  at  Buffalo.  To 
design  the  experiments,  we  held  working  sessions 
which  included  airline  inspection  representatives 
(through  the  ATA)  and  FAA  Technical  Center 
representatives.  This  group  met  formally  on  two 
occasions  during  1994  at  AANC  facility  in 
Albuquerque;  the  research  team  performed  its  detailed 
design  work  outside  these  meetings. 

Reliability  of  NDI  for  crack  detection  is  typically 
reported  as  one  or  more  Probability  of  Detection  (POD) 
curves,  plotted  against  crack  length.  As  the  design 
progressed,  it  became  obvious  to  the  research  team  that 
visual  inspection  was  a  multifaceted  activity;  unlike 
NDI  of  cracks,  it  could  not  be  characterized  by  a  series 
of  performance  curves  plotted  against  a  single 


characteristic.  While  an  equivalent  curve  can  be 
generated  for  visual  inspection  for  the  single  defect  type 
of  crack,  as  Table  9.1  (previous  page)  shows,  it  would 
only  give  a  partial  description  of  inspection 
performance.  Thus  the  goals  of  VIRP  were  defined  as 
follows: 

A.  To  establish  probabilities  of  detection  for  a  range 
of  different  types  of  visual  inspection  (cracks, 
corrosion,  wear  and  tear,  and  mechanical)  for  a 
"typical”  aircraft  visual  inspection. 

B.  To  provide  quantified  "best  practice"  guidance  on 
improving  visual  inspection  reliability. 

A  research  program  was  developed  based  on  these 
goals.  This  process  has  been  described  fully  in  the 
research  team’s  1994  White  Paper  on  VIRP  and  is  only 
summarized  here. 

The  VIRP  experiments  are  designed  to  achieve  Goals  A 
and  B  (above)  in  a  series  of  experiments.  Because  of 
the  large  number  of  factors  potentially  affecting 
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performance,  a  single  experiment  cannot  economically 
provide  a  measure  of  overall  performance  and 
simultaneously  quantify  the  effects  of  important 
parameters.  Thus,  the  program  was  developed  as  a 
Benchmark  Experiment  (Goal  A),  followed  by  a  series 
of  Follow-On  Studies  giving  parametric  measures  of 
various  factors  of  interest  (Goal  B). 

The  detailed  protocols  for  the  Benchmark  Experiment 
were  partly  based  upon  AANC’s  1992-94  study  of 
human  reliability  in  eddy  current  inspection  (Spencer, 
et  ak,  1994),  Because  the  main  vehicle  for  testing  was 
AANCs  high-cycle  Boeing-737,  that  aircraft  had  to  be 
subjected  to  a  thorough  inspection  to  determine 
potential  indications/defects.  This  was  performed  in  a 
Baseline  study  during  1994,  using  qualified  commercial 
inspection  personnel  to  perform  a  D-check  package  on 
the  fuselage  structure.  This  study’s  findings  were  placed 
into  a  database  that  could  be  accessed  either  by  the  job 
card  (workcard)  on  which  the  defect  was  found  or  by 
the  defect  type.  This  database  was  used  to  develop  a 
new  set  of  job  cards  specific  to  VIRP,  each  containing 
known  defects.  These  job  cards  were  often  designed  as 
subsets  of  the  original  job  cards  so  as  to  include  specific 
areas  and  specific  defects  of  most  interest. 

To  determine  the  factors  to  be  included  in  the 
experimental  program's  design,  the  working  group 
(ATA,  FAA,  and  research  team)  listed  factors  known  or 
suspected  to  affect  inspection  performance  under  four 
headings  (see  Czaja,  Drury,  &  Shealy,  1981): 

o  Task:  The  actions  the  inspector  performs,  for 
example:  which  defects  are  inspected  for,  the  level 
of  inspection,  the  time  constraints,  etc. 

©  Operator:  Individual  characteristics  of  the 
inspector,  such  as  visual  ability,  training, 
motivation,  familiarity  with  the  task. 

o  MacMne:  Details  of  the  structure  inspected  and  of 
the  tools  used,  from  mirrors  and  flashlights  to 
layout  of  the  job  card. 

©  Enviroiiiiient:  The  surroundings  of  the  inspection 
task.  This  obviously  includes  visual,  thermal,  and 
auditory  environments,  but  can  also  include 
restrictiveness  of  access  and  even  managerial 
climate. 

Based  on  these  considerations,  the  working  group 
decided  that  the  Benchmark  experiment  would  be 
concerned  primarily  with  using  the  factors  to  ensure  that 


results  would  be  representative  of  industry  practice.  The 
Follow-On  experiments  would  then  examine  specific 
factors  one  or  two  at  a  time.  In  this  way,  any  data 
obtained  in  the  Follow-On  experiments,  e.g.,  new 
flashlight  designs  or  better  training,  could  be  compared 
directly  against  the  Benchmark  study  to  measure  the 
effectiveness  of  any  changes  in  inspection  "best 
practice." 

9.3.1  Benchmark  Study 

During  the  benchmark  study,  a  group  of  inspectors, 
who  have  not  seen  the  test  aircraft  previously,  will  be 
asked  to  make  a  visual  inspection  of  specific  areas 
defined  by  the  VIRP  jobcards.  The  benchmark  will  be 
set  up  as  a  "typical"  scenario  by  controlling  key 
variables.  Each  inspector  will  inspect  a  number  of  areas 
of  the  aircraft  in  order  to  assess  that  inter-inspector 
reliability.  Videotapes  of  inspectors  performing 
inspection  tasks  will  be  made.  Following  the  actual 
aircraft  inspection,  each  inspector  will  be  interviewed 
using  a  structured  interview  schedule  to  elicit  his  or  her 
expert  judgments  about  the  factors  influencing 
successful  performance.  Analysis  of  the  results  will 
include  consideration  of  the  types  of  errors  inspectors 
may  make.  The  outputs  of  the  benchmark  study  will  be 
as  follows: 

Quantitative  Results 

1 .  probabilities  of  detection  for  different 
flaw/defect  types  and  sizes 

2.  inter-inspector  reliability 

3.  estimate  of  the  effects  of  inspector 
characteristics  included  in  the  design 
(see  below) 

Use  of  videotape  as  a  recording  medium  will  allow  a 
classification  of  whether  an  unreported  defect  was  due 
to  an  inspector  not  reacting  to  the  defect  (search 
failure),  or  reacting,  but  deciding  not  to  report  it 
(decision  failure).  After  this  experiment,  it  will  be 
possible  to  measure  the  reliabilities  of  the  search 
process  and  of  the  decision  process  so  that  detailed 
guidance  can  be  given  on  suitable  improvement 
interventions. 

Both  factors  to  be  varied  in  this  experiment  concern 
difficulty  of  the  task.  Job  cards  were  developed  to 
provide  inspection  tasks  with  either  high  or  low 
physical  access  difficulty  and  with  high  or  low  visual 
complexity.  Twelve  experienced  airline  inspectors, 
recruited  through  the  ATA  members,  will  inspect  each 
area  of  the  B-737  test  bed  over  a  two-and-a-half-day 
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period  (Figure  9.1,  next  page).  They  will  also  inspect  a 
sample  of  the  crack  test  panels  developed  for  the  NDI 
eddy-current  reliability  experiment  (Figure  9.2,  page 
173)  to  determine  how  reliable  inspectors  are  on  a 
highly-controlled,  but  realistic,  task  of  the  aircraft. 

Factors  to  be  fixed  were  chosen  so  that  they  would  be  at 
the  "best  practice"  level.  Thus,  only  experienced 
inspectors  will  be  used.  Each  will  use  a  good  standard 
tool  kit  (mirror,  flashlight,  etc.),  and  the  jobcards  will 
be  well-designed  (Patel,  Drury  &  Lofgren,  1994),  The 
hangar  environment  is  low-noise  with  minimum 
distractions,  and  the  support  stands  are  sturdy  and  of  the 
correct  height. 

In  addition  to  the  primary  data  of  whether  or  not  each 
inspector  detected  each  defect,  secondary  data  will  be 
available  from  a  video  debriefing  procedure.  This 


procedure  prompts  inspectors  to  describe  what  they 
were  doing,  and  why  they  were  doing  it,  during  various 
inspection  procedures.  The  procedure  we  will  use  is 
called  a  Retrospective  Verbal  Protocol  (e,g.,  Ohnemus 
&  Biers,  1993).  It  provides  valuable  insight  into  the 
cognitive  mechanisms  of  inspection  (e.g.,  Kleiner, 
Drury,  Sharit,  &  Czaja,  1989).  To  improve  the 
precision  of  the  experiment  and  to  obtain  a  greater 
understanding  of  individual  factors  in  aircraft  visual 
inspection,  a  small  battery  of  tests  will  be  given  to  each 
subject.  These  tests,  which  provide  co- variates  for  later 
analysis,  include  visual  performance,  mechanical 
comprehension,  and  field  dependence  (e.g.,  Thackray, 
1992;  Drury,  &  Wang,  1986). 

As  of  March  1995,  a  pilot  subject  has  been  tested,  and 
the  lessons  learned  were  incorporated  into  the 
Benchmark  Study.  Ten  test  subjects  have  now  been  run. 
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9.3.2  FoI!oW“On  S  todies 

While  a  large  variety  of  studies  are  possible  following 
the  benchmark  study,  only  those  of  most  direct  benefit 
to  the  user  community,  e.g.,  to  FAA  and  ATA,  will  be 
performed  as  part  of  the  VIRP.  The  developed 
protocols  and  the  characterized  B-737  test  bed  could  be 
used  as  the  basis  for  specific  commercial  studies  in  a 
manner  similar  to  AANC's  continuing  work  in  NDL  No 
follow-on  studies  will  be  finalized  until  the  results  of 
the  benchmark  study  are  available;  indeed,  the  design 
of  the  follow-on  studies  is  likely  to  be  an  ongoing 
activity  of  the  group  as  industry  and  FAA  needs  are 
better  defined. 

In  the  White  Paper  produced  before  the  Benchmark 
Study  began,  we  identified  four  potential  follow-up 
studies: 

1.  Effects  of  fatigue  and  rest  pauses  on  the  detection  of 
flaws 

Objective:  To  assist  in  providing  guidance  on 
the  effective  use  of  rest  pauses  or  other  work 
changes  to  enhance  inspection  and  to  combat 
the  effects  of  fatigue. 

Background:  Studies  of  human  reliability  in  other 
domains  have  shown  that,  with  fatigue/time  on  shift,  the 
performance  of  experts  tends  to  deteriorate;  in  extreme 
instances  performance  reverts  to  that  of  relatively 
untrained  personnel.  Studies  have  also  clearly  related 
the  ability  to  detect  signals  to  levels  of  attentiveness. 
The  negative  effects  of  both  of  these  factors  may  be 
controlled  with  rest  pauses.  Data  from  this  study  could 
be  compared  with  that  from  the  benchmark  study. 

2.  Perceptual  factors 

Objective:  To  form  a  basis  for  guidance  on 
suitable  lighting  levels,  color  enhancements, 
etc.,  needed  to  design  an  appropriate  physical 
environment  for  visual  inspection  tasks. 
Background:  Visual  detection  will  be 
influenced  by  pertinent  factors  in  the  physical 
environment  such  as  contrast,  color 
enhancement,  light  levels,  etc.  Job  aids  such 
as  flashlights,  mirrors,  etc.,  will  interact  with 
such  factors.  Aspects  such  as  the  color  of  the 


inspection  surface  may  affect  ease  of 
detection. 

3.  Search  criteria 

Objective:  To  study  the  effects  of  search 
criteria  on  the  probability  of  detecting  flaws 
and  to  assist  in  the  development  of  guidance 
on  suitable  search  criteria. 

Background:  The  ability  to  detect  signals  has 
been  shown  to  be  dependent  on  the  search 
criteria  provided,  e.g.,  general  versus  detailed 
inspection.  Factors  such  as  the  number  of  type 
of  flaws  to  be  searched  for  may  influence  the 
probability  of  detection  of  both  these  and 
other  types  of  flaws. 

4.  Decision  criteria 

Objective:  To  study  the  effects  of  decision 
criteria  on  the  probability  of  detection  of  flaws 
and  to  provide  guidance  on  suitable  decision 
criteria. 

Background:  The  criteria  provided  to  or 
assumed  by  inspectors  will  influence  both  the 
hit/miss  and  false  alarm  rates.  Criteria  may 
also  be  affected  by  the  actual  or  perceived 
consequences  of  calling  or  failing  to  call  a 
flaw. 


The  VCRP  is  designed  to  respond  directly  to  industry 
needs,  as  expressed  through  the  ATA,  and  to  FAA 
concerns.  Over  the  first  year  a  test  bed  has  been 
characterized,  protocols  developed,  and  job  cards 
produced  so  that  subsequent  studies  will  benefit  in 
terms  of  reduced  design  time  and  effort.  As  the 
Benchmark  study  is  completed  and  analyzed  (Spring, 
1995),  benefits  in  data  handling  and  analysis  for 
subsequent  studies  will  also  be  available.  The  whole 
VIRP  effort  has  been  unique  in  the  way  it  has  combined 
knowledge  of  human  inspection  behavior,  experience  of 
aircraft  inspection,  and  statistical  design  of  experiments. 
Future  experiments  will  extend  the  VIRP  effort  to 
investigate  the  effects  of  inspector  fatigue,  the  visual 
environment,  and  for  the  criteria  used  by  the  inspector. 
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10.0  introduction 


In  an  earlier  review  of  studies  and  programs  dealing 
with  nondestructive  inspection  (NDI)  reliability,  a 
repeated  finding  was  the  existence  of  large  individual 
differences  among  inspectors  in  their  inspection  profi¬ 
ciency  (FAA/AAM  &  GSC,  1993).  The  few  studies 
cited  in  this  review  that  attempted  to  determine  possible 
reasons  for  these  differences  in  NDI  proficiency  were 
generally  unsuccessful. 

While  the  above  review  was  confined  largely  to  NDI 
reliability  in  the  Air  Force  and  the  nuclear  power  indus¬ 
try,  a  recent  study  of  commercial  aviation  inspec¬ 
tion/repair  facilities  confirmed  that  inspector-to- 
inspector  differences  were  a  major  source  of  variation 
in  the  commercial  field  as  well  (Spencer  &  Schurman, 
1994).  While  differences  among  facilities  in  the  proce¬ 
dures  used  (or  in  the  training  inspectors  received)  un¬ 
doubtedly  accounted  for  some  of  the  differences  found 
in  this  study,  it  seems  unlikely  that  these  factors  ac¬ 
counted  for  all  of  the  variation  among  inspectors. 

In  the  review  report  noted  above,  research  studies  of 
individual  differences  in  inspection  and  vigilance,  in¬ 
terviews  with  NDI  training  supervisors  and  inspectors, 
and  opinions  of  experts  in  the  NDI  field  suggested  a 
number  of  skills,  aptitudes,  and  traits,  measures  of 
which  might  be  relevant  to  NDI  selection  and/or  profi¬ 
ciency.  To  explore  these  possibilities,  a  study  was  con¬ 
ducted  to  examine  relationships  among  many  of  these 
aptitudes,  traits  and  performance  on  a  simulated  eddy- 
current  inspection  task.  More  specifically,  the  study 
sought  (a)  to  determine  the  relationships  of  various 
predictor  measures  derived  from  these  skills,  aptitudes 
and  NDI  performance  and  (b)  to  examine  evidence  of 
fatigue  changes,  if  any,  over  a  simulated  day-shift  pe¬ 


riod  (Shepherd  &  GSC,  in  press).  ^  In  addition  to  these 
primary  purposes  of  the  study,  a  number  of  other  rela¬ 
tionships  were  also  examined.  A  summary  of  the  major 
findings  follows: 

•  Accuracy  of  inspection  (low  numbers  of 
missed  faults  and  false  alarms)  was  found  to 
be  positively  related  to  test  measures  of  me¬ 
chanical  ability  and  attention-concentration. 

•  Speed  of  inspection  was  positively  related  to 
test  measures  of  such  traits  as  extroversion, 
impulsivity,  and  lack  of  meticulousness. 

•  Accuracy  and  speed  of  inspection  were  found 
to  be  unrelated. 

•  There  were  increases  in  the  percentage  of 
faults  missed  and  in  the  percentage  of  good 
rivets  called  “faulty”  (false  alarms)  both 
within  and  between  performance  sessions 
over  the  simulated  day-shift  period.  Although 
statistically  significant,  these  percentage  in¬ 
creases  were  relatively  small,  ranging  from 
0.8  to  4.5  percent. 

•  Expressed  liking  for  inspection  was  unrelated 
to  performance  (missed  faults,  false  alarms,  or 
speed)  on  the  NDI  task. 

»  There  were  no  differences  between  males  and 
females  in  either  task  performance  or  in  liking 
for  inspection. 


*  A  more  extensive  background  and  rationale  for  the 
predictor  measures  employed,  as  well  as  the  need  for 
further  information  on  possible  fatigue-related  per¬ 
formance  changes,  were  provided  in  the  earlier  study 
and  are  not  reviewed  here. 
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The  present  study  was  conducted  to  follow-up  on  the 
findings  of  this  previous  study?  Of  particular  concern 
was  the  question  of  whether  the  relationships  between 
NDI  task  performance  and  psychometric  measures  of 
mechanical  ability  and  attention-concentration  would 
hold  for  a  different  group  of  subjects  drawn  from  a 
somewhat  different  population.  A  secondary  purpose  of 
this  follow-up  study  was  to  re-examine  a  number  of  the 
relationships  noted  above. 

The  task  employed  in  this  study  was  a  slightly  modified 
version  of  the  computer- simulated  NDI  eddy-current 
task  used  in  the  previous  study.  This  task  was  devel¬ 
oped  by  Drury  and  his  colleagues  at  the  State  Univer¬ 
sity  of  New  York  (SUNY)  at  Buffalo  and  was 
described  in  detail  in  the  previous  study  and  in  studies 
by  Drury,  Prabhu,  Gramopadhye,  and  Latorella  (1991), 
and  Latorella,  Gramopadhye,  Prabhu,  Drury,  Smith, 
and  Shanahan  (1992).  It  utilized  a  SUN  SPARC  work¬ 
station  and  incorporated  a  standard  keyboard  and  opti¬ 
cal  three-button  mouse  as  input  devices.  As  Latorella  et 
al.  (1992)  have  emphasized,  this  task  was  not  devel¬ 
oped  to  devise  a  simulator  that  could  be  used  for  train¬ 
ing  on  actual  NDI  tasks,  nor  was  the  aim  to  develop  a 
task  that  could  be  used  to  measure  absolute  values  of 
the  probability  of  detecting  particular  types  and  sizes  of 
faults.  The  aim  was  to  devise  a  task  that  closely  ap¬ 
proximated  the  characteristics  and  requirements  of 
eddy-current  inspection  tasks  to  enable  laboratory  in¬ 
vestigation  of  factors  that  may  influence  NDI  perform¬ 
ance. 

The  task  modification  refened  to  above  involved  nec¬ 
essary  software  changes  that  did  not  change  the  essen¬ 
tial  nature  of  the  NDI  simulation  but  did  change  some 
of  its  response  characteristics.  A  software  problem 
during  the  previous  study  would  cause  the  system  to 
malfunction  at  times,  with  resulting  loss  of  data.  Cor¬ 
recting  this  problem  resulted  in  a  simulation  with 
somewhat  faster  response  characteristics.  The  effects  of 
these  changed  characteristics  on  task  performance  will 
be  described  in  subsequent  sections. 


10.1.1  Subjects 

total  of  37  subjects,  18  males  and  19  females, 
participated  in  the  study.  Subjects  ranged  in  age 
from  18  to  29  years,  had  normal  visual  acuity  (as  de¬ 
termined  from  an  Orthorater  screening  test),  and  were 
paid  $10.00  an  hour  for  their  participation  through  an 


^  Unless  otherwise  noted,  the  words  "previous  study" 
henceforth  refer  always  to  this  earlier  study. 


existing  Federal  Aviation  Administration  (FAA)  con¬ 
tract.  Most  subjects  were  currently  employed  and  at¬ 
tending  a  junior  college,  a  vocational  institute,  a 
military  training  program,  or  a  local  university  on  a 
part-time  basis.  Educational  levels  ranged  from  high 
school  graduate  to  college  graduate.  Approximately 
one-third  of  the  subjects  were  Air  Force  enlisted  per¬ 
sonnel  assigned  to  Tinker  Air  Force  Base. 

None  of  the  subjects  was  an  aircraft  mechanic  or  in¬ 
spector  and  none  had  prior  training  or  experience  in 
aircraft  maintenance  or  inspection.  As  in  the  previous 
study,  this  ensured  a  more  heterogeneous  sample, 
thereby  maximizing  differences  among  individuals. 

The  inclusion  of  college  students  appeared  justifiable 
on  the  basis  of  several  recent  studies  of  inspection  per¬ 
formance  that  used  both  students  and  inspectors 
(Gallway,  1982;  Gallway  &  Drury,  1986).  The  former 
study  was  reasonably  similar  to  the  present  one  in  that 
it  involved  selection  tests  and  inspection  performance. 
Neither  study  found  any  significant  differences  between 
students  and  inspectors  in  the  comparisons  made.  Fi¬ 
nally,  educational  levels  in  the  present  study  were  com¬ 
parable  to  those  of  inspectors  in  the  recent  field  study 
of  NDI  reliability  conducted  by  Sandia  (Spencer  & 
Schurman,  1994). 

10.1.2  Apparatus 

The  basic  apparatus  consisted  of  a  SUN  SPARC  Model 
4/50GX-16-P43  workstation,  a  19-inch  color  monitor, 
and  a  3-button  optical  mouse.  Although  the  nature  of 
the  task  and  its  physical  characteristics  have  been  de¬ 
scribed  in  the  previous  study  and  elsewhere  (Drury  et 
al.,  1991;  FAA/AAM  &  GSC,  1994;  Latorella  et  al., 
1992),  task  elements  are  briefly  reviewed  here. 

The  display  consisted  of  four  basic  task  elements 
(windows).  These  are  shown  in  Figure  10.1  (next  page) 
and  described  in  the  following  sections. 

10.1.2.1  Inspection  Window 

The  lower  left  portion  of  the  screen  displayed  the  in¬ 
spection  window  and  contained  the  actual  rivets  to  be 
inspected.  Although  it  was  possible  to  present  more 
than  one  six-rivet  row  of  rivets  to  the  subject,  only  a 
single  row  was  used  in  this  study.  Each  subject  used 
an  optical  mouse  to  move  the  cursor  around  the  cir¬ 
cumference  of  each  simulated  rivet.  The  subject  was 
free  to  examine  the  rivet  until  he  or  she  decided 
whether  or  not  a  crack  was  present.  If  the  subject  de¬ 
cided  that  a  rivet  was  defective,  he  or  she  pressed  the 
right  mouse  button,  causing  a  red  cross  to  appear  over 
the  "defective"  rivet;  the  words  "rivet  marked  bad" 
appeared  on  the  screen.  If  the  subject  decided  that  a 
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rivet  was  nondefective,  he  or  she  pressed  the  middle 
button,  causing  the  words  "rivet  marked  good"  to  ap¬ 
pear  on  the  screen.  If  a  subject  realized  that  he  or  she 
made  an  incorrect  response,  it  could  be  corrected  by 
pressing  the  appropriate  button. 

When  all  of  the  six  rivets  had  been  inspected,  the  sub¬ 
ject  clicked  the  left  mouse  button  on  the  directional 
block  labeled  "right."  This  caused  a  black  marker  ring 
to  circle  the  last  rivet  inspected,  and  the  next  six  rivets 
in  the  row  appeared  in  the  inspection  window. 

1 0.1 .2.2  MacrO‘View  and  Directionals 

A  macro- view  in  the  upper  left  portion  of  the  screen 
displayed  a  side  view  of  the  aircraft  fuselage  and  the 
row  of  rivets  being  inspected.  Since  only  a  small  por¬ 
tion  of  this  row  was  being  inspected  at  any  given  time 
during  the  task,  the  subject  could  move  the  cursor  over 
the  words  "Where  am  I?"  in  this  area  and  a  momentary 
circle  would  appear  over  the  portion  of  the  rivet  row 
currently  being  examined. 

10.1.2.3  Eddv-Current  Meter 

The  upper  right  portion  of  the  display  contained  a 
simulated  analog  meter  that  served  as  the  eddy-current 
output  indicator.  Deflections  beyond  a  set  point  on  the 
meter  produced  an  audible  signal.  Meter  deflections 
could  be  caused  by: 


•  touching  a  rivet  edge  with  the  cursor  or  mov¬ 
ing  the  cursor  over  the  head  of  a  rivet 

•  the  cursor  passing  over  a  crack,  all  of  which 
were  "subsurface"  and  invisible 

•  the  cursor  passing  over  or  near  simulated  cor¬ 
rosion,  scratches,  or  paint  chips.  (These  were 
simulated  by  2  mm  jagged  lines  at  random  lo¬ 
cations  adjacent  to  a  rivet.)  Not  all  rivets 
contained  such  "noise,"  and  no  rivet  contained 
more  than  one  such  noise  spot. 

10.1.2.4  Lower  Right  Window 

The  lower  right  portion  of  the  display  could  be  used  by 
the  subject  to  exercise  a  number  of  options  (e.g.,  to 
“zoom”  to  take  a  closer  look  at  a  rivet  being  inspected, 
to  stop  the  task  in  order  to  take  a  break,  or  to  display 
elapsed  time).  The  only  feature  used  in  the  present 
study  caused  a  number  to  appear  on  each  rivet  and  was 
used  only  by  the  experimenter  during  training  feedback 
sessions  to  enable  location  and  rechecking  of  rivets  in¬ 
correctly  classified. 

10.1.3  Crack  and  Meter  Characteristics 

As  was  noted  earlier,  the  developers  of  this  task  never 
intended  it  to  be  used  as  a  simulator  for  NDI  training  or 
to  measure  absolute  values  of  the  probability  of  detect¬ 
ing  particular  types  and  sizes  of  faults.  Their  aim  was  to 
develop  a  task  that,  by  approximating  the  characteris- 
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tics  and  requirements  of  eddy-current  inspection  tasks, 
could  be  used  in  the  laboratory  to  investigate  factors 
that  may  influence  NDI  performance.  Nevertheless,  to 
provide  as  much  realism  as  possible,  the  range  (14  to 
350  mils)  and  mean  (approximately  100  mils)  of  fault 
sizes  employed  were  designed  to  correspond  with  those 
that  might  be  encountered  in  the  field  and  approxi¬ 
mated  those  derived  from  data  reported  in  the  recent 
Sandia  eddy-current  reliability  study  (Schurman  & 
Spencer,  1994).  Meter  deflection  was  proportional  to 
crack  size,  with  the  simulated  needle  showing  a  similar 
rapid,  abrupt  deflection  when  the  cursor  passed  over  or 
was  in  close  proximity  to  either  cracks  or  noise  ele¬ 
ments. 

10.1.4  Predictors  and/or  Task  Correlates 

The  previous  study  identified  a  number  of  variables, 
measures  of  which  showed  significant  relationships  to 
performance  on  the  NDI  task  or  appeared  to  warrant  re¬ 
examination.  A  few  of  the  tests  and  measures  used  in 
the  earlier  study  failed  to  correlate  with  any  of  the  per¬ 
formance  criteria  and  were  discarded.  The  variables 
retained  included  measures  of  the  following: 

®  Mechanical  Aptitude 
O  Attentiveness/Distractibility 
®  Extroversion/Impulsivity 
a  Motivation/Perseverance 
9  Decision  Time/ Accuracy 

The  tests  and  measures  used  for  each  of  these  were  dis¬ 
cussed  in  detail  in  the  previous  study.  For  purposes  of 
review,  however,  those  employed  in  this  study  are 
briefly  described  in  the  following  sections. 

1Q.1.4.1  Subjective  Rating  Scale  (SRS) 

This  is  a  simple  self-rating  scale  that  the  author  devel¬ 
oped  and  has  used  in  numerous  studies  (e.g.,  Thackray, 
Bailey,  &  Touchstone,  1977;  Thackray  &  Touchstone, 
1991)  to  assess  current  feeling  levels,  with  measures 
generally  taken  before  and  after  periods  of  task  per¬ 
formance.  The  basic  instrument  consists  of  five  9-point 
scales  measuring  the  dimensions  of  attentiveness,  tired¬ 
ness,  strain,  interest,  and  annoyance.  One  additional 
scale  measuring  effort  required  to  remain  attentive 
during  task  performance  was  also  included.  Although 
the  previous  study  failed  to  show  significant  relation¬ 
ships  of  these  measures  to  task  performance,  this  scale 
was  retained  so  as  to  allow  comparisons  of  feeling 
states  of  subjects  used  in  the  two  studies. 

1Q.1.4.2  Bennett  Mechanical  Comprehension  Test 

One  of  the  recommendations  of  the  Southwest  Re¬ 
search  Institute  study  of  ways  to  improve  NDI  techni¬ 


cian  proficiency  was  to  select  individuals  who  scored 
high  on  mechanical/electronics  aptitude  (Schroeder, 
Dunavant,  &  Godwin,  1988).  This  recommendation 
was  also  echoed  in  interviews  with  NDI  instructors; 
they  believe  that  individuals  who  are  above  average  in 
mechanical  aptitude  make  better  inspectors  (Shepherd 
&  GSC,  in  press).  The  previous  study  found  that  the 
Bennett  Mechanical  Comprehension  Test,  a  measure  of 
the  ability  to  perceive  and  understand  relationships  of 
physical  forces  and  mechanical  elements  in  practical 
situations,  shows  a  significant  relationship  to  perform¬ 
ance;  individuals  scoring  higher  on  the  test  were  more 
accurate  in  their  performance  on  the  NDI  task.  This 
was  the  most  promising  test  result  found  in  the  previous 
study,  and  there  was  a  definite  need  to  re-examine  this 
finding  in  the  follow-up  study. 

1Q.1.4.3  Typical  Experiences  Inventory 

The  ability  to  resist  distraction,  if  it  can  be  measured, 
would  appear  to  have  at  least  face  validity  in  selecting 
inspectors  (Wiener,  1975).  The  Typical  Experiences 
Inventory  is  a  scale  developed  for  use  in  several  previ¬ 
ous  studies  (Pearson  &  Thackray,  1970;  Thackray, 
Jones,  &  Touchstone,  1973).  It  consists  of  a  series  of 
statements  designed  to  measure  ability  to  work  under 
conditions  of  (a)  time  stress,  (b)  threat  of  failure,  (c) 
distraction,  (d)  social  stress,  and  (e)  physical  stress.  In 
the  previous  study,  the  subscale  measure  of  distraction 
susceptibility  showed  a  significant  relationship  to  atti¬ 
tudes  towards  inspection,  i.e.,  individuals  expressing 
dislike  of  inspection  tasks  scored  higher  in  distraction 
susceptibility.  Because  of  this  finding,  it  was  decided  to 
include  this  scale  in  the  follow-up  study. 

1Q.1 .4.4  Arithmetic  and  Digit  Span  Tests  of  the  WechsiM 
Adult  Intelligence  Scale  (WAIS) 

Scores  on  three  subtests  of  the  WAIS  (the  Arithmetic, 
Digit  Span,  and  Digit  Symbol  subtests)  have  been 
shown  in  numerous  factor  analytic  studies  to  measure  a 
factor  that  has  been  variously  named  “Freedom  from 
Distractibility,”  “Attention-Concentration,”  or 
“Concentration-Speed”  (e.g.,  Goodenough  &  Karp, 
1961 ;  Karp,  1963).  In  the  previous  study,  a  factor 
analysis  found  that  the  Arithmetic  and  Digit  Span,  but 
not  the  Digit  Symbol,  loaded  highly  on  the  same  factor 
that  included  the  Bennett  Mechanical  Comprehension 
Test.  Consequently,  the  Arithmetic  and  Digit  Span 
subtests  were  retained  in  the  present  study  to  verify  the 
earlier  findings. 

1Q.1 .4.5  Eysenck  Personality  inventory  (EPI) 

The  Eysenck  Personality  Inventory  is  a  short  inventory 
that  measures  extroversion  and  neuroticism.  As  indi¬ 
cated  in  the  previous  study,  extroversion  has  been 
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studied  extensively  in  the  context  of  vigilance  research 
because  of  the  hypothesis,  originally  formulated  by  Ey¬ 
senck  (1967),  that  extroverts  should  have  more  fre¬ 
quent  lapses  of  attention  and  hence  more  omission 
errors  than  introverts.  Reviews  of  the  use  of  this  per¬ 
sonality  dimension  in  vigilance  research  (Berch  & 
Kantor,  1984;  Wiener,  1975)  have  lent  some  support  to 
the  belief  that  extroverts  generally  do  not  perform  as 
well  on  vigilance  tasks  as  introverts.  Much  less  re¬ 
search  has  been  conducted  on  personality  variables  in 
the  area  of  inspection,  and  no  studies  of  extroversion 
and  inspection  performance  had  been  conducted  at  the 
time  of  Wiener’s  1975  review. 

In  the  factor  analysis  of  the  previous  study,  extrover¬ 
sion  failed  to  load  on  the  factor  correlated  with  per¬ 
formance  errors,  but  did  load  positively  on  Factor  1, 
which  was  the  factor  correlating  significantly  with 
speed  of  inspection.  These  findings  led  to  the  decision 
to  include  the  Eysenck  Test  in  order  to  re-examine  re¬ 
lationship  of  extroversion  to  performance. 

10.1.4.6  Matching  Familiar  Figures  Test  (MFFT) 

The  MFFT  is  a  test  developed  by  Kagan  and  his  asso¬ 
ciates  (Kagan,  Rosman,  Day,  Albert,  &  Phillips,  1964) 
and  consists  of  a  series  of  12  “stimulus”  pictures,  each 
of  which  is  associated  with  8  “response”  pictures.  Ex¬ 
cept  for  the  one  correct  picture  in  each  of  the  response 
sets,  all  differ  from  the  stimulus  picture  in  some  minute 
detail.  Subjects  point  to  the  picture  they  believe  to  be 
the  correct  one  in  each  set  and  continue  to  point  until 
the  correct  one  is  identified.  Both  time  to  first  response 
and  number  of  errors  are  scored.  According  to  the 
authors,  the  test  measures  a  cognitive  style  known  as 
reflection-impulsivity.  Those  who  make  quick,  inaccu¬ 
rate  decisions  on  the  test  are  said  to  have  an  impulsive 
cognitive  style;  those  who  make  slow,  accurate  deci¬ 
sions  are  said  to  have  a  reflective  cognitive  style. 

The  previous  study  found  a  significant  inverse  relation¬ 
ship  between  MFFT  error  scores  and  scores  on  the 
WAIS  Arithmetic  scale,  i.e.,  high  scores  on  the  latter 
scale  were  associated  with  few  errors  on  the  MFFT. 
Because  the  Arithmetic  scale  loaded  on  the  same  factor 
as  the  Bennett  Mechanical  Comprehension  Test,  it 
seemed  desirable  to  re-examine  these  relationships  in 
the  follow-up  study. 

10.1.4.7  Jackson  Personality  Research  Form  (PRF) 

The  Jackson  Personality  Research  Form  (Jackson, 
1974)  is  a  widely  used  test  designed  to  yield  a  set  of 
scores  for  personality  traits  broadly  relevant  to  the 
functioning  of  individuals  in  a  wide  variety  of  situa¬ 
tions.  It  is  a  personality  test  that  focuses  primarily  upon 


areas  of  normal  functioning,  rather  than  psychopathol¬ 
ogy. 

The  Form  E  used  in  this  study  consists  of  sixteen 
scales,  of  which  four  were  re-examined  in  the  follow¬ 
up  study.  The  included  scales  were  (a)  Endurance,  (b) 
Cognitive  Structure,  (c)  Change,  and  (d)  Impulsivity.  A 
brief  description  of  each  and  the  reason(s)  for  its  inclu¬ 
sion  are  as  follows: 

•  Endurance  A  measure  of  the  willingness  to 
work  long  hours  and  to  be  patient  and  unre¬ 
lenting  in  work  habits.  This  was  included  as  a 
possible  measure  of  intrinsic  motivation  or 
perseverance  in  task  performance. 

•  Cognitive  Structure  A  measure  of  the  need  to 
make  meticulous  decisions  based  upon  defi¬ 
nite  knowledge  with  a  dislike  of  ambiguity 
and  uncertainty.  It  was  felt  that  this  trait  might 
be  positively  related  to  search  time,  i.e.,  the 
time  spent  in  searching  each  rivet  for  possible 
faults. 

•  Change  A  liking  for  new  and  different  expe¬ 
riences,  with  a  dislike  and  avoidance  of  rou¬ 
tine  activities.  Inclusion  of  this  trait  is  self- 
evident,  since  NDI  tasks  are  so  often  referred 
to  as  boring  and  monotonous. 

•  Impulsivity  A  measure  of  the  tendency  to  act 
on  the  “spur  of  the  moment”  and  without  de¬ 
liberation.  This  was  included  as  an  additional 
measure  of  impulsivity  to  be  compared  with 
the  impulsivity  measure  derived  from  the 
MFFT. 

Three  of  the  above  scales  (Endurance,  Cognitive 
Structure,  and  Impulsivity)  were  retained  in  the  follow¬ 
up  study  because  they  showed  high  loadings  on  the 
factor  (Factor  1)  of  the  previous  study  that  was  corre¬ 
lated  with  speed  of  inspection.  The  “Change”  scale 
failed  to  correlate  significantly  with  any  of  the  criterion 
measures  of  the  previous  study,  but  was  included  to  re¬ 
examine  its  possible  relationship  to  expressed  dislike  of 
inspection  tasks. 

10.1.4.8  Figure  Preference  Test 

This  test  is  a  paired  comparison  version  of  the 
Munsinger  and  Kessen  (1964)  test  of  preference  for 
complex  versus  simple  perceptual  stimuli.  Subjects 
choose  which  pair,  of  a  set  of  66  pairs  of  figure  draw¬ 
ings  that  differ  in  complexity,  they  prefer.  A  recent 
study  of  industrial  workers  determined  that  preference 
for  simple  stimuli  on  this  test  was  related  to  preference 
for  repetitive,  unchanging  work  requiring  a  constant  fo¬ 
cus  of  attention  (Rzepa,  1984).  Although  this  test  failed 
to  correlate  significantly  with  any  of  the  criterion 
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measures  of  the  previous  study,  it  did  show  a  signifi¬ 
cant  relationship  to  measures  of  distraction  susceptibil¬ 
ity  and  was  retained  as  a  further  possible  measure  of 
attitude  toward  inspection. 

10.1.5  Procedure 

Upon  arrival,  subjects  were  given  a  brief  description  of 
the  purpose  of  the  research  and  signed  an  informed 
consent  form.  The  various  tests  and  measures  forming 
the  predictor  battery  were  then  administered.  Following 
completion  of  this  phase,  subjects  received  practice 
sessions  in  the  use  of  the  mouse,  were  required  to  read 
and  be  tested  on  a  document  describing  eddy-current 
testing  and  the  need  for  it,  and  then  began  performance 
training. 

The  initial  phase  of  training  began  with  practice  in  use 
of  the  computer  mouse.  This  was  accomplished  with  a 
display  program  consisting  of  a  single  simulated  rivet 
head  with  a  training  circle  surrounding  it.  Subjects 
practiced  using  the  mouse  and  cursor  to  circle  the  rivet 
while  staying  within  the  circle.  After  each  pre-selected 
block  of  training  trials,  feedback  was  provided  consist¬ 
ing  of  average  times  required  to  circle  the  rivet,  and  av¬ 
erages  of  the  number  of  times  the  cursor  head  touched 
the  rivet  or  went  outside  the  circle.  Training  continued 
until  the  subject  reached  a  consistent  level  of  perform¬ 
ance.  This  usually  required  10  to  20  minutes  of  prac¬ 
tice. 

Training  on  the  inspection  task  consisted  of  three  sepa¬ 
rate  training  sessions,  each  60  rivets  long.  Thirty  per¬ 
cent  of  the  rivets  in  each  training  session  contained 
faults  (cracks).  In  addition,  the  second  and  third  ses¬ 
sions  also  contained  small,  but  visible  (2  mm)  “noise” 
spots  at  various  locations  at  or  near  a  rivet.  Frequency 
of  “noisy  rivets”  was  also  thirty  percent.  Location  of 
faults  and  noise  was  randomly  assigned  for  each  task 
session  (both  training  and  subsequent  test  tasks).  Per¬ 
formance  feedback  was  automatically  provided  after 
each  block  of  10  rivets.  In  the  first  session,  training 
circles  were  provided  around  each  rivet  to  assist  the 
subject  in  keeping  the  cursor  in  the  appropriate  region 
while  circling  the  rivets;  no  training  circles  were  used 
in  the  second  and  third  sessions. 

Following  a  noon  lunch  break,  subjects  performed  two 
300-rivet  task  sessions.  These  sessions  were  self-paced, 
and  task  duration  for  each  subject  varied  from  a  mini¬ 
mum  of  about  60  minutes  to  the  maximum  allowable 
duration  of  90  minutes.  There  was  a  scheduled  15  mi¬ 
nute  rest  break  between  each  session,  although  subjects 
were  told  they  could  take  short  (10-20  second) 

“stretch”  breaks  as  needed  during  any  session.  No 


feedback  was  provided  following  the  task  sessions,  and 
the  frequency  of  both  faults  and  noise  was  held  at  30 
percent  each. 

Subjective  rating  scales  were  administered  at  the  be¬ 
ginning  and  end  of  each  task  session.  At  the  end  of  the 
second  session,  subjects  were  debriefed  and  asked  sev¬ 
eral  questions  about  their  performance.  These  included 
questions  about  how  well  they  thought  they  had  per¬ 
formed,  and  whether  they  felt  that  inspection  was  a  type 
of  work  that  they  could  see  themselves  doing  or  would 
choose  to  do  on  an  everyday  basis. 


10.2.1  Task  Performance 

10.2.1.1  Performance  Measures:  Reliability. 
Intercorrelations,  and  General  Observations 

hree  performance  measures  were  derived  from  the 
NDI  inspection  task:  (a)  percentage  of  faults 
missed,  (b)  percentage  of  good  rivets  marked  faulty 
(false  alarms),  and  (c)  mean  time  per  rivet.  Of  the  two 
types  of  error  (failing  to  detect  a  faulty  rivet  or  calling  a 
good  rivet  bad),  missed  faults  were  more  common.  On 
the  average,  approximately  7.8%  of  faulty  rivets  were 
missed,  while  only  about  1 .2%  of  good  rivets  were 
marked  faulty.  The  percentage  of  false  alarms  was 
comparable  to  the  2%  obtained  in  the  previous  study 
and  to  false  alarm  rates  found  in  the  recent  Sandia/FAA 
study  (Schurman,  1993).  The  percent  faulty  rivets 
missed,  however,  was  considerably  less  than  the  23% 
missed  in  the  previous  study.  The  most  reasonable  ex¬ 
planation  for  this  difference  between  the  two  studies 
involves  the  software  modifications  to  the  NDI  simula¬ 
tion  that  were  mentioned  earlier.  These  changes,  by 
eliminating  most  of  the  previous  slight  lag  in  meter  re¬ 
sponse,  apparently  increased  the  likelihood  that  faults 
would  be  detected.  Test  trials  conducted  by  the  author 
following  the  software  modifications  confirmed  that  the 
change  in  meter  characteristics  did,  indeed,  increase  the 
probability  of  fault  detection. 

The  two  measures  of  performance  error  (percent 
missed  faults  and  percent  false  alarms)  were  found  to 
be  positively  correlated  (r  ~  .50,  p  <  .01),  but  neither 
was  significantly  related  to  speed  of  inspection  (p  > 
.01).  The  lack  of  a  relationship  between  speed  of  in¬ 
spection  and  measures  of  performance  error  was  con¬ 
sistent  with  findings  of  the  previous  study.  However, 
the  significant  correlation  between  missed  faults  and 
false  alarms  was  not  anticipated,  since  the  previous 
study  found  them  to  be  unrelated.  Examination  of  the 
score  distributions  for  these  two  variables  revealed  that 
they  appeared  generally  unrelated,  except  for  three  in- 
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dividuals  who  had  exceptionally  high  false  alarm  rates 
and  who  were  also  above  average  in  missed  faults.  In¬ 
clusion  of  these  individuals  may  have  biased  the  rela¬ 
tionship,  resulting  in  a  correlation  that  was  spuriously 
high.  A  nonparametric  measure  (the  Spearman  rank  or¬ 
der  correlation)  computed  for  these  two  variables  failed 
to  reach  significance  (^>.01),  suggesting  that  this 
measure  may  better  approximate  the  true  relationship 
between  missed  faults  and  false  alarms  for  this  particu¬ 
lar  set  of  data. 

1 0.2.1 .2  Performance  Change  Across  Sessions 

One  of  the  purposes  of  the  previous  study  was  to  exam¬ 
ine  the  data  for  any  evidence  of  fatigue  changes  during 
the  morning  and  afternoon  sessions.  While  examination 
of  possible  fatigue  effects  was  not  a  principal  concern 
of  this  follow-up  study,  the  earlier  study  had  shown 
some  evidence  of  fatigue-related  performance  changes, 
and  it  was  decided  to  compare  performance  change 
over  the  two  test  sessions.  Mean  values  for  each  per¬ 
formance  variable  are  shown  in  Table  10.1. 

Analyses  of  variance  revealed  a  significant  increase  in 
percent  missed  faults  (F(l/35)=70.7,  p<.01)  and  a  sig¬ 
nificant  decrease  in  mean  time  per  rivet  (F(l/35)=42.5, 
P<.01).  Percent  false  alarms  showed  no  significant 
change  (F<1 .00). 

The  changes,  although  statistically  significant  for  2  of 
the  3  measures,  were  relatively  small  and  generally  in 
accord  with  the  findings  of  the  previous  study.  Also 
consistent  with  the  earlier  study  was  the  finding  of  no 
gender  differences  in  performance  levels  or  change 
across  sessions.  Consequently,  gender  is  not  shown  as  a 
variable  in  the  table. 


10.2.2  Rating  Scale  Variables 

10.2.2.1  Pre-  to  Post-Task  Changes 
Measures  of  attentiveness,  tiredness,  strain,  interest, 
and  annoyance  were  obtained  for  each  subject  at  the 
beginning  and  end  of  the  two  performance  sessions.  An 
additional  item  administered  only  at  the  end  of  the  per¬ 
formance  sessions  required  subjects  to  rate  the  effort 
required  to  maintain  alertness  when  the  sessions  began 
and  when  they  ended.  Mean  pre-  and  post- task  values 
for  each  rating  variable  are  shown  in  Table  10.2.  Sepa¬ 
rate  analyses  of  variance  revealed  significant  pre-  to 
post-task  decreases  in  attentiveness  (F(l/36)=36,6, 
P<.01)  and  interest  (F(l/36)-64.4,  p<.01),  along  with 
significant  increases  in  tiredness  (F(l/36)=27.2,  p<.01), 
annoyance  (F(  l/36)=9. 1 ,  p<.01),  and  effort 
(F(l/36)“30.5,  p<.01).  The  increase  in  strain  shown  in 
Table  10.2  was  not  significant  (F(l/36)=3.8,  p>.01). 

Pre-session  ratings  indicated  that  subjects  began  each 
session  feeling  moderately  attentive,  somewhat  above 
their  normal  energy  level,  moderately  relaxed,  moder¬ 
ately  interested,  and  not  annoyed.  Since  all  variables 
were  rated  on  9-point  scales,  with  5  representing  the 
midpoint  or  average  value  for  each  feeling  state,  it  is 
apparent  that  post-session  levels  for  all  variables  were 
near  or  below  this  midpoint  value.  Thus,  subjects  could 
not  be  characterized  as  inattentive,  tired,  strained, 
bored  or  annoyed  following  the  performance  sessions. 

Ratings  of  perceived  effort  indicated  that  slight  effort 
was  required  to  maintain  involvement  in  the  task  ini¬ 
tially,  with  moderate  effort  required  towards  the  end  of 
a  task  session. 

Initial  levels  of  all  the  rating  variables,  as  well  as  the 
magnitude  and  direction  of  changes,  were  remarkably 


Session 

Performance  Variables 

1 

2:.. 

Session  IJns 

Percent  Faults  Missed 

5.19 

10.14 

7.80 

Percent  False  Alarms 

1.15 

1.19 

1.17 

Mn  Time  Per  Rivet  (sec) 

12.36 

10.86 

11.61  1 

Mn  Post-Session  Ratings 

Attentiveness 

7.1 

5.7 

Tiredness 

3.9 

5.3 

Strain 

3.4 

3.9 

Interest 

6.9 

5.0 

Annoyance 

1.2 

1.8 

Effort 

3.1 

4.6 

183 


Individual  Differences  in  NDI  Performance 


Chapter  10 


similar  to  those  obtained  in  the  previous  study.  This 
clearly  indicates  that  the  samples  used  in  both  studies 
were  comparable  in  terms  of  their  initial  feelings  and 
attitudes,  as  well  as  in  changes  that  occurred  resulting 
from  task  performance. 

10.2.3  Predictor  Variables  and  Performance 

A  number  of  exploratory  analyses  were  conducted  us¬ 
ing  factor  analysis  solved  for  3  to  5  factors.  The  clear¬ 
est  relationships  were  found  using  a  principal 
components  analysis  with  varimax  rotation  and  solved 
for  3  factors.  Loadings  of  each  predictor  variable  on 
the  3  factors  are  shown  in  Table  10.3.  A  cut-off  crite¬ 
rion  of  .60  was  again  used  to  select  those  variables 
contributing  to  factor  interpretation.  This  means  that  a 
variable  would  have  to  explain  at  least  36%  of  a  fac¬ 
tor's  variance  in  order  for  it  to  be  included  in  a  factor's 
interpretation.  The  factors  were  identified  with  labels  as 
follows: 

©  Factor  I  -  Mechanical  Aptitude  This  factor 
appears  to  stand  alone  as  an  ability  factor,  in 
contrast  to  the  other  factors  which  represent 
personality  dimensions.  Three  tests  loaded 
substantially  on  this  factor:  The  Bennett  Me¬ 
chanical  Comprehension  Test  and  the  WAIS 
Arithmetic  subtest  showed  high  positive 
loadings,  while  the  MFFT  error  score  showed 
a  high  negative  loading.  The  Bennett  Test 
would  seem  to  define  the  factor,  while  the 
other  two  suggest  important  attentional  com¬ 
ponents  associated  with  it. 
o  Factor  2  -  Tirelessness/Patience  Scales 

loading  positively  on  this  factor  (PRF  Cogni¬ 
tive  Structure  and  PRF  Endurance)  suggest  a 
meticulous,  unfaltering  personality  style, 
while  the  negative  loading  on  the  PRF  Impul- 
sivity  scale  suggests  deliberation  and  patience. 


®  Factor  3  -  Extroversion/Experience  Seeking 
This  factor  is  characterized  by  high  loadings 
on  the  EPI  Extroversion  Scale  and  the  PRF 
Change  Scale.  Taken  together,  these  two 
scales  would  appear  to  identify  an  outgoing 
personality  dimension  with  a  dislike  and 
avoidance  of  routine  activities. 

Pearson  product  moment  coixelations  between  each 
factor  score  and  the  various  performance  criterion 
measures  showed  only  one  of  the  factors  to  be  signifi¬ 
cantly  related  to  performance.  Factor  1,  which  had 
substantial  positive  loadings  on  both  the  Bennett  Me¬ 
chanical  Comprehension  Test  and  the  WAIS  Arithme¬ 
tic  subtest,  and  a  negative  loading  on  the  Matching 
Familiar  Figures  Test  error  score,  was  negatively  corre¬ 
lated  with  missed  faults  (r=-.62,  p<.01)  and  with  false 
alarms  (r=-.53,  p<.01).  Unlike  the  previous  study,  the 
present  study  found  speed  of  inspection  (mean 
time/rivet)  to  be  unrelated  to  any  of  the  factors. 

Both  the  present  and  previous  studies  find  a  significant 
relationship  between  a  measure  of  mechanical  compre¬ 
hension  (the  Bennett  Mechanical  Comprehension  Test) 
and  performance  accuracy.  This  is  interesting  for  sev¬ 
eral  reasons.  One  reason  is  that  it  is  consistent  with  one 
of  the  recommendations  of  the  Southwest  Research 
Institute  study  of  ways  to  improve  NDI  technician  pro¬ 
ficiency.  That  recommendation,  based  mostly  on 
speculation,  was  to  select  individuals  for  NDI  who 
scored  high  on  mechanical/electronics  aptitude 
(Schroeder,  Dunavant,  &  Godwin,  1988).  NDI  instruc¬ 
tors  also  believe  that  individuals  who  arc  above  aver¬ 
age  in  mechanical  aptitude  make  better  inspectors 
(FAA/AAM  &  GSC,  1993).  The  Bennett  Mechanical 
Comprehension  Test,  as  indicated  in  the  manual  for  this 
test,  has  been  validated  on  various  groups  of  aircraft 
employees,  with  validity  coefficients  ranging  from  .52 
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Factor  j 
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2 
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Typical  Experiences  Inventory 

0.071 

-0.281 

0.537 

Bennett  Mech  Comp  Test 

0.649 

0.142 

0.388 

Match  Fam  Fig  Error 

-0.736 

0.037 

0.292 

Match  Fam  Fig  Time 

0.405 

0.087 

-0.507 

EPI  Extroversion 

0.221 

-0.184 

0.676 

WAIS  Digit  Span 

0.465 

0.194 

0.043 

WAIS  Arithmetic 

0.823 

0.091 

0.131 

PRF  Change 

0.025 

0.257 

0.672 

PRF  Cog  Structure 

0.058 

0.710 

-0.110 

PRF  Endurance 

0.171 

0.780 

0.016 

PRF  Impulsivity 

-0.148 

-0.824  ' 

0.107 

Figure  Preference 

-0.580 

-0.092 

0.414 
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to  .62.  These  groups  have  included  shop  trainees  and 
aircraft  factory  workers  in  mechanical  jobs  (Bennett, 
1969).  The  findings  of  both  the  present  and  previous 
study  suggest  that  the  Bennett  test  may  be  a  useful 
predictor  of  NDI  performance,  as  well.  This  would 
support  the  above-noted  recommendation  of  the 
Southwest  Research  Institute,  as  well  as  the  opinions 
expressed  by  NDI  instructors,  of  the  relationship  be¬ 
tween  mechanical  ability  and  NDI  performance. 

The  other  two  tests  loading  on  Factor  1  were  the 
Arithmetic  subtest  of  the  WAIS  and  the  error  score  of 
the  Matching  Familiar  Figures  Test.  With  regard  to  the 
first  of  these,  several  factor  analytic  studies  have  shown 
the  WAIS  Arithmetic  and  Digit  Span  subtests  and,  less 
frequently,  the  WAIS  Digit  Symbol  subtest  to  load  on  a 
factor  that  has  been  variously  named  “Freedom  from 
Distractibility”  or  “Attention-Concentration” 
(Goodenough  &  Karp,  1961 ;  Karp,  1963).  In  the  pre¬ 
vious  study,  the  Digit  Span  subtest  loaded  on  the  factor 
containing  the  Bennett,  while  in  the  present  study  the 
Arithmetic  subtest  showed  the  highest  loadings  on  this 
factor.  Both  studies,  then,  found  evidence  of  an  addi¬ 
tional  dimension  (attention-concentration)  that  was  re¬ 
lated  to  NDI  task  performance.  As  mentioned  in  an 
earlier  section  of  this  paper,  studies  by  Gallwey  (1982) 
and  Wang  and  Drury  (1989)  have  also  found  a  rela¬ 
tionship  of  these  attention-concentration  subtests  to  in¬ 
spection  performance.  Wang  and  Drury,  however, 
noted  that  while  a  measure  such  as  the  WAIS  Digit 
Span  correlated  with  performance  errors  in  some  of  the 
inspection  tasks  studied,  it  failed  to  correlate  in  others. 
The  authors  concluded  that  the  relationships  of  WAIS 
subtest  measures  of  attention-concentration  to  inspec¬ 
tion  performance  may  have  to  be  empirically  deter¬ 
mined  for  different  inspection  tasks. 

The  other  variable  with  a  high  loading  on  Factor  1  was 
the  MFFT  error  score,  which  loaded  negatively  on  this 
factor.  The  Matching  Familiar  Figures  Test  is,  accord¬ 
ing  to  its  developers,  a  measure  of  the  cognitive  style 
known  as  reflcction-impulsivity  (Kagan  et  ah,  1964); 
those  making  quick,  inaccurate  decisions  on  this  test 
are  said  to  have  an  impulsive  cognitive  style,  while 
those  who  are  more  deliberate  and  accurate  are  said  to 
have  a  rellective  style.  The  high  negative  loading  of  the 
MFFT  error  measure  shown  in  Table  10.3  (previous 
page),  taken  in  conjunction  with  the  lower,  but  positive 
loading  on  the  MFFT  time  measure,  suggests  that  in¬ 
dividuals  who  were  slow  and  accurate  in  their  perform¬ 
ance  on  the  MFFT  also  tended  to  be  more  accurate  in 
their  performance  on  the  simulated  NDI  task.  However, 
since  the  MFFT  did  not  show  significant  loadings  on 
the  mechanical  comprehension  factor  in  the  previous 


study,  the  validity  of  this  apparent  relationship  to  NDI 
task  performance  is  questionable. 

10.2.4  Gender,  Liking  for  Inspection,  and  Self  Es¬ 
timates  of  Task  Performance 

During  the  debriefing  period,  subjects  were  asked 
whether  they  thought  they  might  like  inspection  work 
or  could  visualize  themselves  as  an  inspector.  They 
were  told  that  the  NDI  task  they  just  completed  repre¬ 
sented  only  one  type  of  inspection  activity  and  that  they 
should  try  to  base  their  answer  on  inspection  jobs  in 
general.  The  answers  were  coded  “1”  if  inspection  ap¬ 
pealed  to  them  and  “2”  if  it  did  not.  This  variable  was 
then  correlated  with  the  predictor  measures  and  with 
performance.  Like  the  findings  of  the  previous  study, 
the  variable  “liking”  was  not  significantly  related  to 
any  of  the  factor  scores  or  with  any  measure  of  per¬ 
formance  (/?>.01).  The  lack  of  a  relationship  between 
liking  for  inspection  and  actual  task  performance  is 
consistent  with  findings  of  Summers  (1984)  in  his  fol¬ 
low-up  study  of  the  early  Air  Force  “Have  Cracks,  Will 
Travel”  study  (Lewis  et  al.,  1978).  Summers  found  no 
relationship  between  expressed  liking  for  (or  dislike  ot) 
inspection  among  Air  Force  technicians  and  actual  NDI 
performance. 

As  with  the  previous  study,  there  was  an  apparent  gen¬ 
der  difference  in  attitudes  toward  inspection,  with 
males  showing  a  greater  liking  for  inspection  and  fe¬ 
males  a  greater  dislike.  These  data  are  shown  in  Table 
1 0.4.  A  chi-square  test,  however,  revealed  the  obtained 
gender  differences  to  be  nonsignificant  {ji  >  .01).  Al¬ 


,lEernaiaptep!§ssiripi^B 

Gender 

Like 

Inspection 

Dislike 

Inspection 

Males 

11 

7 

Females 

7 

12 

though  not  related  to  liking  for  inspection  and,  as  noted 
above,  not  related  to  any  performance  measures,  gen¬ 
der  was  significantly  conelated  (r  =  -.62,  /?  <  .01 )  with 
scores  on  the  Bennett  Mechanical  Comprehension  Test. 
As  with  the  previous  study,  males  tended  to  score 
higher  than  females.  This  finding  is  entirely  consistent 
with  normative  data  published  for  the  test  (Bennett, 
1969)  and  was  expected.  However,  because  of  the 
substantial  loadings  of  this  lest  on  the  factor  (Factor  1 ) 
which  was  significantly  correlated  with  performance 
accuracy,  an  indirect  relationship  of  gender  to  perform¬ 
ance  is  suggested. 
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During  debriefing,  subjects  were  also  asked  to  evaluate 
how  well  they  thought  they  performed  relative  to  others 
performing  the  same  inspection  task.  Twenty-seven  of 
the  37  subjects  felt  their  performance  was  about  the 
same  as  most,  nine  felt  that  it  was  better,  and  only  one 
subject  believed  his  performance  to  be  worse  than 
most.  Separate  f-tests  were  conducted  to  compare  the 
performance  (missed  faults  and  false  alarms)  of  sub¬ 
jects  believing  their  performance  was  better  than  most 
with  those  who  thought  it  was  about  the  same.  None  of 
the  comparisons  yielded  significant  (p>  .0\)  t  values, 
showing  that  perceptions  of  performance  were  unre¬ 
lated  to  actual  performance.  The  lack  of  a  relationship 
between  self-ratings  of  inspection  performance  and 
actual  NDI  performance  is  in  accord  with  similar  find¬ 
ings  of  the  earlier  Air  Force  NDI  study  (Summers, 
1984)  noted  above. 


A  previous  study  examined  the  relationships  among 
a  number  of  predictor  tests  and  measures  and  per¬ 
formance  on  a  simulated  eddy-current  inspection  task 
(Shepherd  &  GSC,  in  press).  The  tests  and  measures 
employed  were  intended  to  tap  various  skills,  aptitudes, 
and  traits  that  research  studies  of  inspection,  interviews 
with  NDI  training  supervisors  and  inspectors,  and 
opinions  of  experts  in  the  NDI  field  had  suggested 
might  be  relevant  to  NDI  proficiency  (Shepherd  & 
GSC,  in  press).  While  the  obtained  relationships  be¬ 
tween  a  number  of  the  predictor  measures  and  task  per¬ 
formance  were  encouraging,  findings  were  considered 
to  be  tentative  until  validated  in  a  subsequent  study  us¬ 
ing  a  different  group  of  subjects. 

The  study  reported  here  was  conducted  to  follow-up 
the  earlier  results.  The  basic  approaches  of  the  two 
studies,  including  the  procedures  followed  and  task 
employed,  were  essentially  the  same.  Except  for  the 
fact  that  a  different  group  of  subjects  was  used,  the 
major  differences  between  this  study  and  the  previous 
one  were  that  (a)  fewer  predictor  measures  were  em¬ 
ployed,  since  those  showing  no  promise  in  the  previous 
study  were  eliminated  and  (b)  the  task  sessions  were 
shorter,  as  examination  of  possible  fatigue  effects  was 
not  a  principal  concern  of  the  follow-up  study.  A  sum¬ 
mary  and  comparison  of  the  principal  common  findings 
of  the  two  studies  follows: 

o  Both  studies  were  consistent  in  finding  a  sig¬ 
nificant  relationship  between  scores  on  the 
Bennett  Mechanical  Comprehension  Test  and 
performance  accuracy  on  the  simulated  NDI 
task,  i.e.,  higher  scores  on  the  Bennett  Test 
were  associated  with  more  accurate  NDI  task 


performance.  This  finding  was  the  single  most 
important  of  the  two  studies  and  supports  the 
beliefs  and  opinions  of  NDI  experts  that  me¬ 
chanical  aptitude  may  be  a  good  predictor  of 
NDI  proficiency. 

o  Both  studies  were  consistent  in  finding  a  sig¬ 
nificant  relationship  between  NDI  task  per¬ 
formance  accuracy  and  scores  on  WAIS 
measures  of  attention-concentration.  In  the 
previous  study,  the  WAIS  Digit  Span  subtest 
showed  the  greater  relationship,  while  in  the 
follow-up  study  it  was  the  WAIS  Arithmetic 
subtest. 

o  The  follow-up  study,  but  not  the  earlier  one, 
found  an  apparent  relationship  between 
MFFT  error  scores  and  performance  accu¬ 
racy.  Because  of  this  lack  of  consistency  be¬ 
tween  studies,  the  validity  of  this  relationship 
is  uncertain. 

o  There  were  statistically  significant  increases 
in  the  percentage  of  faults  missed  during  the 
task  sessions  in  both  studies.  This  increase 
occurred  over  the  simulated  day  shift  of  the 
earlier  study  and  during  the  shorter  afternoon 
sessions  of  the  follow-up  study.  The  increase 
in  percentage  of  faults  missed,  however,  was 
relatively  small  in  both  studies  and  may  not  be 
of  practical  significance. 

o  The  two  studies  agreed  in  finding  no  relation¬ 
ship  between  gender  and  either  liking  for  in¬ 
spection  or  performance  on  the  simulated 
NDI  task. 

o  Liking  for  inspection  was  found  to  be  unre¬ 
lated  to  task  performance  in  both  studies. 

o  No  relationship  existed  between  speed  of  in¬ 
spection  and  performance  in  either  study. 
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This  report  is  divided  into  four  sections.  In  the  first 
section,  Background  and  Literature  Review,  we 
review  state-of-the-art  literature  on  team  training.  In 
the  next  section,  we  outline  a  general  framework  for 
considering/evaluating  tasks’  potential  for  team 
training,  also  identifying  team  training  strategies  for 
improving  different  team  competencies.  In  the 
section  on  Team  Training  for  Aircraft  Inspection 
Maintenance,  we  outline  implications  of  team 
training  for  aircraft/inspection  tasks  and  report  results 
of  a  study  evaluating  effectiveness  of  team  training 
for  an  aircraft  maintenance  task.  In  the  final  section, 
Team  Training  for  A  &  P  Schools,  we  describe  how 
team  training  could  be  incorporated  in  an  A  &  P 
school  curriculum  and  provide  a  functional 
description  of  a  computer-based  team  training  tool. 
We  performed  this  project  in  close  cooperation  with  a 
major  maintenance  repair  facility  and  an  A  &  P 
school  so  that  results  address  the  aviation 
community’s  concerns. 


11.1.1  Introduction 


Previous  FAA  reports  on  human  factors  in  aviation 
maintenance  (Shepherd,  1991;  FAA,  1993)  have 
recognized  the  importance  of  training.  To  this  point, 
training  for  aircraft  maintenance  and  inspection 
systems,  essentially,  has  aimed  at  improving 
individual  skills  (Shepherd  and  Parker,  1990), 
ranging  from  improving  diagnostic  skills  through 
aircraft  maintenance  training  (Johnson,  1990(a))  to 
acquiring  and  enhancing  visual  inspection  skills  to 
improve  airframe  structural  inspection  (Shepherd, 
1993;  Gramopadhye  et  al.,  1992).  Researchers  have 
tended  to  concentrate  on  improving  the  overall 
training  program  either  with  training  methodology 


(e.g.,  Drury  and  Gramopadhye,  1990;  Desormiere, 
1990)  or  with  the  training  delivery  system’s 
technology  for  on-the-job  training,  classroom 
training,  tutoring,  and  computer-based  training 
(Gordon,  1994;  Johnson  et  al.,  1992;  Drury  et  al..  In 
Press).  While  there  has  been  much  study  of 
individual  skills,  there  has  been  little  on  developing 
team  skills. 

Task  analysis  of  aircraft  inspection  and  maintenance 
activities  (Shepherd,  1990)  reveals  that  the  aircraft 
maintenance/inspection  system  is  complex,  requiring 
above-average  coordination,  communication,  and 
cooperation  among  inspectors,  maintenance 
personnel,  supervisors,  and  members  of  other 
subsystems-planning,  stores,  and  shops-to  be 
effective  and  efficient.  Many  maintenance  activities 
technicians  or  inspectors  undertake  can  be  performed 
more  effectively  and  efficiently  with  a  team.  Though 
the  airline  industry  widely  recognizes  advantages  of 
teamwork  (Hackman,  1990),  individual  AMTs,  not 
the  teams  they  work  with,  are  held  responsible  for 
faulty  work.  The  individual  AMT  licensing  process 
and  concerns  about  personal  liability  often  result  in 
AMTs  and  supervisors  being  unwilling  to  share 
knowledge  and  responsibility  across  shifts  or  with 
less-experienced,  less-skilled  colleagues.  This 
problem  is  exacerbated  by  the  fact  that  experienced 
inspectors  and  mechanics  are  retiring  and  are  being 
replaced  with  a  younger,  less-experienced  workforce. 
The  newer  AMTs  lack  the  knowledge  and  skills  of 
the  experienced  AMTs  they  replace  and  also  are  not 
trained  to  work  as  a  team  member. 

The  FAA  continually  addresses  the  problem  of 
individual  development  of  initial  AMT  skills.  The 
newly  established  Part  66  of  the  FAR  specifically 
addresses  significant  technological  advancements  in 
the  aviation  industry,  as  well  as  the  past  decade’s 
advancements  in  training  and  instructional 
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methodologies.  The  FAA,  through  its  Office  of 
Aviation  Medicine,  has  funded  efforts  to  develop 
advanced  training  tools  for  future  AMTs.  New 
training  technologies  under  development,  e.g., 
intelligent  tutoring  systems  and  embedded  training, 
will  be  available  to  A  &  P  training  schools. 
Application  of  new  training  technologies  should  help 
reduce  the  gap  between  AMTs’  current  skills  and 
those  skills  necessary  to  maintain  advanced  systems. 

The  effort  invested  in  developing  individual  skills 
has  led  to  a  revised  FAR,  to  new  training  tools  (e.g., 
Johnson,  1990(b);  Johnson  1992)  applying  advanced 
technology,  and  to  development  of  advanced  training 
delivery  systems  (Gramopadhye,  Drury  and  Prabhu, 

In  Press).  The  area  now  needing  attention  is 
development  of  team  skills.  In  addition  to 
fundamental  skills,  today’s  employers  require 
creativity,  an  ability  to  communicate,  and  an  ability 
to  work  in  a  team.  Team  skills  are  often  not  well- 
developed  or  part  of  the  background  of  AMTs  now 
joining  the  workforce.  The  problem  is  made  more 
urgent  since  the  aviation  maintenance  workforce  is 
much  younger  and  less-experienced,  usually  without 
experience  working  on  military  aircraft.  The  younger 
workforce  does  not  carry  the  passion  for  airplanes 
older  workers  expect.  An  FAA  report  (FAA,  1991) 
stated,  "People  today  Join  airlines  for  many  reasons 
beyond  the  love  of  planes.  This  clear  shift  plus  other 
changes  in  labor  work  force  confound  the  long- 
service  employee.  Older  employees  are  somewhat 
dismayed  with  the  newer  mechanics'  acquired  skills, 
their  lassez-faire  attitude,  and  their  high  turnover." 

Inspectors  and  maintenance  technicians  are 
challenged  to  work  autonomously  while  being  part  of 
a  team.  In  a  typical  maintenance  environment,  an 
inspector  looks  for  and  reports  defects.  A 
maintenance  person  repairs  the  reported  defect  and 
works  with  the  original  inspector  or  the  buy-back 
Inspector  to  ensure  that  work  meets  standards.  During 
the  repair  process,  inspectors  and  maintenance 
technicians  work  as  a  team  with  colleagues  from  the 
same  and  the  next  shift,  as  well  as  with  personnel 
from  areas  like  planning  or  stores,  to  ensure  that  the 
task  is  completed  (FAA,  1991).  In  any  typical 
maintenance  environment,  a  technician  must  learn  to 
be  a  team  member,  to  communicate,  and  to 
coordinate  activities  with  other  technicians  and 
inspectors.  However,  AMTs  joining  the  workforce 
lack  team  skills.  The  current  A  &  P  curriculum  often 
encourages  students  to  compete,  so  that  new  AMTs 
often  are  not  prepared  to  work  cooperatively.  To 


prepare  student  AMTs  for  workplace  realities,  we 
need  to  find  new  ways  to  build  students’ 
technological,  interpersonal,  and  sociotechnical 
competence  while  incorporating  team  training  and 
communication  skills  into  the  curriculum. 

The  present  study’s  general  objective  was  to  present 
the  importance  of  teamwork  and  team  training  in  the 
aircraft  inspection  environment  by  focusing  on  teams 
and  strategies  to  improve  team  performance.  We 
expected  results  to  help  prepare  new  AMTs  for 
teamwork  in  the  aircraft  inspection  environment.  The 
study’s  specific  objectives  were  the  following: 

o  To  understand  the  role  of  teamwork  and  team 
training  in  the  aircraft  inspection/maintenance 
environment 

®  To  evaluate  the  effectiveness  of  a  team  training 
activity  with  AMTs  from  an  A  &  P  school 
®  To  develop  guidelines  and  suggestions  for 

incorporating  team  training  in  the  A  &  P  school 
curriculum 

©  To  use  results  obtained  from  earlier  activities  to 
develop  functional  specifications  for  a  computer- 
based  tool  for  team  training. 

To  ensure  that  our  project  addressed  the  aviation 
community’s  needs,  we  conducted  the  project  in 
cooperation  with  a  major  aircraft  repair  and 
overhauling  facility  and  with  an  FAA-licensed  A  &  P 
school. 

11.1.2  Literature  on  Teams 

Teams  have  received  a  great  deal  of  attention  in 
recent  research  literature  (Salas,  et  al.,  1992;  Driksell 
and  Salas,  1992;  Glickman,  et  al.,  1987).  There  is 
consensus  among  those  who  study  industrial  and 
organizational  behavior  that  teams/work  groups  will 
be  the  cornerstone  of  future  American  industry 
(Cannon-Bowers  et  al.,  1992;  Cummings,  1981;  Shea 
and  Guzzo,  1987).  Teamwork  will  be  essential 
because  tomorrow’s  task  demands  are  likely  to 
exceed  individual  capabilities;  hence,  individuals  will 
need  to  work  together  more.  Teamwork  will  assume 
a  critical  role  for  achieve  desired  performance.  Due 
to  inherent  complexities  of  studying  teams  in 
organizations,  the  abundant  literature  is  fragmented, 
incomplete,  and  often  contradictory.  However,  it  is 
important  to  glean  from  past  work  any  findings  that 
can  help  us  understand  teamwork,  team  performance, 
and  strategies  for  improving  team  skills. 
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The  review  of  the  team  literature  that  follows  is 
limited  to  the  objectives  of  this  study  and  to  a  greater 
extent  restricted  to  teams  who  perform  in  a  complex 
and  dynamic  environment  similar  to  the  environment 
of  aircraft  inspection/maintenance,  which  takes  place 
at  sites  ranging  from  those  of  large  international 
carriers,  through  startup  and  regional  airlines,  to  the 
fixed  based  operators  associated  with  general 
aviation  (Drury  et  al.,  1990).  Previous  FAA  reports 
detail  the  complexity  of  the  aircraft 
inspection/maintenance  environment,  clearly 
indicating  above  average  coordination,  cooperation 
and  communication  necessary  to  accomplish  tasks. 
Additionally,  the  importance  of  teams  has  been 
emphasized  in  the  National  Plan  for  Aviation  in 
Human  Factors  (FAA,  1991),  where  both  the  industry 
and  government  groups  agreed  that  additional 
research  needs  to  be  conducted  to  evaluate  teamwork 
in  the  aircraft  maintenance/inspection  environment. 

11.1.3  Team  and  Teamwork  Defined 

A  definition  of  what  constitutes  a  team  facilitates  our 
discussion  on  teams  in  the  aircraft  inspection  and 
maintenance  environment.  Throughout  the  literature, 
team  and  teamwork  are  defined  differently.  The 
following  definition  of  team  is  consistent  with  the 
nature  of  the  effort  required  for  aircraft 
inspection/maintenance  tasks  (Morgan  et  al.,  1986 
p6):  “a  team  is  a  distinguishable  set  of  two  or  more 
individuals  who  interact  interdependently  and 
adaptively  to  achieve  specified,  shared  and  valued 
objectives.”  A  number  of  principles  have  been 
proposed  to  ensure  that  teams  work  effectively  in  any 
situation.  Scholtes  (1992)  suggests  that  effective 
teamwork  depends  on  the  following  ten  essential 
ingredients: 

1 .  Clarity  in  team  goals 

2.  An  improvement  plan 

3.  Clearly  defined  roles  of  team  members 

4.  Clear  communication 

5.  Beneficial  team  behavior 

6.  Well-defined  decision  procedures 

7.  Balanced  participation 

8.  Established  ground  rules 

9.  Awareness  of  the  group  process 

10.  Use  of  scientific  approach. 

For  teams  to  be  effective,  its  members  must  work 
collectively  to  achieve  the  overall  task  objective.  To 
accomplish  an  objective,  some  sort  of  task 
dependency  must  exist  among  team  members. 
According  to  Salas  et  al.  (1992),  the  completion  of  a 
task  objective  necessitates  the  following; 


a)  exchange:  dynamic  exchange  of  information  and 
resources  among  team  members 

b)  coordination:  coordination  of  different  task 
activities  and  adjustments  to  changes  in  task  structure 

c)  organizational  structure:  some  sort  of 
organizational  structure  of  members. 

Research  in  team  and  teamwork  has  shown  that 
training  facilitates  the  entire  team  process  (Glickman 
et  al.,  1987;  Salas  et  al.,  1992;  Swezey  and  Salas, 
1992). 

Most  literature  on  teams  in  the  aviation  industry  has 
focused  on  the  CRM  (Crew  Resources  Management) 
training  program,  which  focuses  on  cockpit  training 
for  air  crews  (FAA,  1993;  Helmreich,  et  al.,  1989; 
Helmreich  and  Wilheim,  1991;  Foushee  and  Manos, 
1981).  CRM  typically  encompasses  several  team 
concepts,  including  team  communication  skills, 
interaction,  situational  awareness,  assertiveness,  and 
leadership  skills.  Although  CRM  programs  have 
existed  for  more  than  a  decade,  there  has  been  only 
limited  use  of  the  programs  for  maintenance  and 
inspection  crews.  To  date,  little  research  has 
evaluated  teams  working  in  the  aircraft  maintenance 
environment.  However,  since  they  realize  the 
importance  of  teams,  several  aircraft  carriers  and 
repair  facilities  have  developed  in-house  training 
programs.  These  programs  often  are  part  of  larger 
management  training  programs,  focusing  on  teaching 
management  and  non-management  personnel  to 
improve  safety  and  efficiency  (e.g.,  Robertson  et  al, 
1994;  Taggart,  1990).  They  are  not  specifically 
developed  for  maintenance  and  inspection  personnel. 

11.1.4  Team  Evolution 

To  understand  how  training  can  provide  measurable 
changes  in  team  behavior  that  enhance  the  efficiency 
and  effectiveness  of  teamwork  in  aircraft 
maintenance,  we  must  examine  the  evolution  of 
teams.  Then  we  can  develop  effective  intervention 
strategies  that  can  impact  teamwork.  In  recent  years, 
several  conceptual  frameworks  and  theories  have 
been  proposed  to  explain  the  team-evolution  process. 
In  this  section,  we  review  salient  frameworks  and 
theories,  drawing  upon  previous  researchers’  work  to 
develop  a  new  framework  for  understanding  the  team 
process  in  the  aircraft  maintenance  environment.  The 
theories  described  below  are  only  representative;  our 
aim  in  including  them  is  to  explain  team  performance 
and  training. 

Hackman’s  (1983)  normative  model  offers  a 
comprehensive  conceptualization  of  group  process  in 
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the  organizational  environment.  Though  the  model  is 
not  developed  for  a  highly  structured  team,  it 
emphasizes  organizational  input  and  the  effort,  skills, 
and  strategies  of  team  members  bring  to  accomplish 
team  goals.  Gersick  (1988)  described  a  time  and 
transition  model  for  teams,  focusing  on  the  dynamic, 
evolving  nature  of  team  performance.  The  model 
shows  how  exchange  of  information  and  resources 
among  team  members  can  result  in  effective  team 
performance.  In  Gladstein’s  (1984)  Group 
Effectiveness  Model,  group  effectiveness  is  a 
function  of  different  group  processes,  such  as 
communication  and  strategy  discussions,  moderated 
by  group  task  demands,  such  as  task  complexity  and 
environmental  uncertainty.  This  is  one  of  the  few 
models  tested  with  a  large  sample  of  teams  in  the 
work  environment.  Morgan  et  al.'s  (1986)  Team 
Evolution  and  Maturation  Model  (TEAM) 
hypothesizes  that  teamwork  develops  through  several 
phases,  beginning  with  loosely  organized  groups  of 
individuals  and  proceeding  to  become  a  highly 
effective  team  over  time.  This  model  conceptualizes 
a  team  as  going  through  developmental  phases  and 
proceeding  from  ineptness  and  exploratory 
interactions  to  the  final  level  of  effective,  efficient 
team  performance.  The  model  considers  two 
distinguishable  types  of  team  activities  through  the 
steps  of  team  evolution:  task-related  activities  and 
team-related  generic  activities.  Task-related  activities 
are  associated  with  developing  operational  skills  to 
perform  technical  tasks;  team-related  activities  are 
involved  in  developing  team  interaction,  e.g., 
relationships,  coordination,  and  interaction. 

Other  models  of  team  performance  emphasize  a  task 
analytic  approach  to  team  training,  e.g.,  Naylor  and 
Dickinson,  1969;  Shiflett  et  al.,  1982.  These  models 
consider  team  performance  as  a  function  of  the  sub¬ 
task  the  team  has  to  perform.  They  imply  that  the 
organization  and  task  complexity  establish  optimal 
work  and  communication  and  interact  to  determine 
individual  and  team  training  requirements  for 
enhanced  team  performance.  Tannenbaum  et  al. 
(1992)  integrate  previously  described  models  in  a 
framework  for  team  performance  and  team  training. 
Canon-Bowers  et  al.  (In  Press)  state  that,  since  teams 
operate  in  diverse  work  environment  performing  a 
wide  variety  of  tasks,  constructs  such  as  teamwork 
and  team  training  can  only  be  understood  in  the 
context  within  which  they  occur.  Tannenbaum  et  al. 
(1992)  proposed  framework  explains  this  context. 


Having  reviewed  various  frameworks  and 

theories,  we  now  propose  our  framework  for 
considering  the  team  process  in  the  aircraft 
maintenance  environment.  Drawing  from  task 
analysis  of  aircraft  inspection  and  maintenance 
operations  (Drury  et  al.,  1990;  FAA,  1991),  from  site 
visits  to  repair  facilities,  from  observations  made 
with  training  personnel  and  A  &  P  school  instructors, 
and  from  a  detailed  review  of  the  team  models,  we 
developed  the  framework  shown  as  Figure  11.1 
(Chapter  1 1  -  Appendix).  This  framework  serves  as  a 
first  step  for  understanding  teamwork  in  aircraft 
inspection  and  maintenance  operations;  it  could  be 
seen  as  an  extension  of  Tannenbaum  et  al.’s  (1992) 
team  effectiveness  model. 

The  framework  illustrates  the  interaction  among 
internal  factors,  external  factors,  the  team  process, 
training  strategies,  and  outcome  measures.  External 
and  internal  factors  effect  the  team  process.  External 
factors  are  categorized  as  follows: 

Organizational  factors:  organization’s  size  ,  type 
(e.g.,  airline,  general  aviation,  repair  facility), 
reward  structure,  management  structure, 
communication  norms,  and  organizational  climate. 

Environmental  factors:  level  of  environmental  stress 
(work  conduct  in  hangars  or  flight-line)  and 
environmental  uncertainty. 

Equipment  factors:  automation,  complexity, 
specialization,  equipment  availability,  and  safety. 

Task  factors:  task  organization  (type  of  aircraft 
check:  A-,  B-,  C-,  or  Heavy-check),  task  type  (e.g., 
avionics,  power  plant,  hydraulics,  sheet  metal, 
frame),  task  complexity,  and  task  structure. 

The  internal  factors,  composed  of  individual  and 
team  skills,  can  be  categorized  as  follows: 

Individual  skills  factor:  This  represents  individual 
team  members’  skills  and  is  best  represented  by 
AMTs’  knowledge,  skills,  and  abilities.  In  an  aircraft 
inspection/maintenance  environment,  the  individual 
skills  factor  is  determined  by  AMTs’  experience 
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working  on  different  aircraft  types  and  with  different 
aircraft  systems. 

Team  skills  factor:  The  team  members’  ability  to 
work  together  productively  is  dependent  on  their 
interpersonal  skills,  on  the  team’s  composition,  on 
the  number  of  people  in  the  team,  and  on  how  long 
members  have  worked  together.  We  identified  team 
skills  relevant  to  aircraft  maintenance  tasks  and 
present  them  in  Table  1  l.l  (Chapter  1 1  -  Appendix). 
The  name  for  each  team  skill  is  based  on  suggestions 
by  Salas  et  al  (1992);  they  were  established  after  a 
comprehensive  review  of  the  literature  on  teams. 
According  to  Morgan  et  al.  (1986),  team  skills  that 
are  isolated  and  identified  can  provide  a  framework 
for  team  performance  assessments.  Although  attitude 
is  not  considered  a  team  skill  dimension  per  se,  it  is  a 
"cognitive"  entity  that  can  be  acquired  through 
training  (Gagne,  1988);  hence,  it  is  shown  separately 
in  Table  11.1  (Chapter  1 1  -  Appendix).  Previous 
studies  have  shown  that  attitude  is  important  for 
teamwork  and  team  performance. 

External  and  internal  factors  impact  team  interaction, 
as  well  as  the  team  process.  However,  team 
development  is  evolutionary:  a  team  matures  over 
time  (Morgan  et  al.,  1986).  When  viewed  in  light  of 
Morgan  et  al.'s  (1986)  TEAM  model,  individual 
skills  reflect  task  behavior  and  represent  team 
members’  abilities  to  perform  assigned  technical 
tasks;  team  skills  reflect  team  members  ability  for 
successful  interaction  and  coordination.  Both  skill 
acquisition  and  team  evolution  can  be  enhanced 
through  training  (Morgan  et  al.,  1987).  Specific  ways 
for  imparting  individual  training  to  AMTs  has  been 
widely  covered  in  the  literature;  hence,  our  effort 
focuses  only  on  team  training. 

AMTs  are  members  of  not  only  one  team,  but  of 
several  teams  working  on  different,  yet  similar  tasks. 
At  an  aircraft  repair  facility,  an  AMT  may  work  on 
different  subsystems  of  various  aircraft  and  with 
different  team  members  over  a  scheduled 
maintenance  period.  For  such  situations,  it  is  critical 
to  identify  generic  skills  (Cannon-Bowers,  et  al.,  In 
Press)  and  to  train  team  members  accordingly. 
Cannon-Bowers  et  al.  refer  to  these  as  "transportable 
team  skills."  At  the  same  time,  training  AMTs  on 
transportable  skills,  in  itself,  may  not  be  sufficient  to 
ensure  successful  team  performance.  For  such 
performance,  AMTs  need  training  on  task-specific 
team  skills,  focusing  on  aircraft  inspection  and 
maintenance  tasks.  Methodology  for  this  type  of 


team  training  is  outlined  in  the  section  on  Team 
Training. 

The  entire  team’s  output  can  be  determined  by 
examining  the  changes  in  measures  of  individual  and 
team  process  and  of  task  performance. 

Individual  process  measures:  These  measures 
identify  changes  in  an  individual's  task  knowledge, 
skills  and  ability  after  he  or  she  takes  part  in  a  team 
activity,  also  reflecting  changes  in  an  individual's 
mental  model  and  understanding  of  an  entire  task. 

Team  process  measures:  These  measures  identify 
evolution  of  new  team  processes  by  changes  in 
members'  specific  team  skills,  i.e.,  coordination, 
communication,  leadership,  and  interpersonal  skills. 

Task  performance  measures:  Performance  of  an 
aircraft  inspection  or  maintenance  task  is  measured 
on  the  dimensions  of  accuracy,  speed,  and  safety. 
Accuracy  measures  the  quality  of  a  job  the  team 
completed.  Speed  measures  time  required  to 
accomplish  a  task.  Safety  refers  to  the  team  members 
ability  to  adhere  to  safety  procedures  by  not 
endangering  themselves  or  other  team  members. 
Measurement  procedures  used  to  evaluate  teams  must 
be  sensitive  to  typical  speed/accuracy  tradeoffs. 

We  used  our  understanding  of  teamwork  to  identify 
specific  strategies  for  training  AMTs  in  A  &  P 
schools.  In  the  following  section,  we  outline  these 
strategies.  Later  in  the  report,  we  identify  specific 
team  projects  which  could  be  incorporated  into  A  & 

P  school  curricula  and  report  results  of  the  study  we 
conducted  to  evaluate  how  team  training  improves 
team  skills  for  an  aircraft  maintenance  task. 


Team  performance  is  a  function  of  the  average 
skills  of  its  members.  Individual  skills  appear  to 
be  a  necessary,  but  not  sufficient,  condition  for 
effective  team  performance;  and  the  correlation 
between  average  skill  level  and  average  team 
performance  is  typically  small  (Bass  and  Barett, 
1981;  Teborg  et  al.,  1976).  According  to  Steiner 
(1972),  team  performance  is  dependent  on  team 
members’  ability  to  perform  assigned  tasks  and  on 
their  ability  to  coordinate  work  flow  and  to 
communicate  effectively.  This  process  can  be 
facilitated  by  team  training. 
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Development  of  a  team  training  program  follows 
classic  training  program  development  methodology. 

It  begins  with  a  thorough  analysis  of  the  training 
program’s  requirements  and  needs  (goals).  The  next 
step  is  establishing  knowledge,  skills  and  abilities 
necessary  for  the  job;  these  are  used  to  specify  the 
training  program’s  behavioral  objectives  forming  the 
basis  for  evaluating  the  training  program.  The 
knowledge,  skills,  and  abilities  cun'ently  required  for 
aircraft  maintenance  does  not  include  team  skills. 
Team  training  is  instruction  team  members  receive  as 
a  unit  to  enhance  team  performance  (Nieva  et  al, 
1978).  It  includes  training  strategies  to  enhance  team 
skills.  When  team  training  must  be  combined  with 
individual  training  in  a  single  program,  research 
shows  team  training  to  be  most  efficient  and  effective 
when  team  members  first  develop  individual  skills. 
Swezey  and  Salas’  (1992)  taxonomy  identifies 
characteristics  of  team  training  to  incorporate  in 
every  training  program  as  communication,  task 
organization,  team  decision-making,  team 
organization,  and  information  transmission.  Specific 
strategies  to  enhance  AMT  team  skills  are  outlined 
below. 

11.3.1  Lecture 

Lecture  is  most  appropriate  for  transportable  team 
skills  and  can  be  used  to  introduce  basics  of  teams, 
teamwork,  and  the  role  of  teams  in  enhancing 
performance.  Lectures  are  most  beneficial  for  team 
organization/collaboration  in  identifying  the  nature  of 
interdependencies  for  team  members  and  developing 
an  understanding  of  the  team’s  structure.  AMTs  can 
be  taught  how  other  members  influence  their 
performance,  what  contributions  other  AMTs  make  , 
the  roles  of  inspectors,  and  cleanup  crews,  and  for 
what  conditions  they  must  adapt  their  performance. 
For  example,  members  should  know  what  to  do  when 
particular  equipment  is  unavailable,  when  a  specific 
inspector  is  not  available  or  when  a  member  is 
assigned  to  a  new  task.  Lecture  can  also  be  used  to 
train  AMTs  in  proper  communication  by  giving 
examples  of  good  and  poor  communication.  AMTs 
can  be  taught  what  type  of  communication- written 
and  oral-they  should  have  with  other  members;  to 
whom  they  must  pass  information,  e.g.,  writing  up  a 
non-routine  workcard  or  passing  work  to  the  next 
shift;  and  from  whom  they  must  receive  instructions. 
Communication  includes  both  technical  and  non¬ 
technical  information.  Team  members  should  be 
trained  on  how  to  provide  and  receive  performance 
feedback  on  individual  and  team  performance  so  that 


individual  members  and  the  team  as  a  whole  use  it  to 
enhance  performance. 

11.3.2  Team  Meetings 

Team  meetings,  i.e.,  group  interaction  methods,  are 
another  popular  technique  (Goldstein,  1986).  This 
consists  of  bringing  AMTs  together  to  interact  in  a 
relatively  unstructured  environment.  Team  meetings 
can  be  effective  for  analyzing  interpersonal  problems 
and  for  developing  effective  understanding  and 
coordination  among  team  members. 

11.3.3  Role-Playing 

Role-playing  can  be  used  for  training  generic  team 
skills.  Members  become  aware  of  each  other’s  roles 
(Cannon-Bowers,  et  al..  In  Press)  by  interacting  with 
each  other  in  role-playing  situations.  They  can  learn 
the  knowledge,  skills,  and  abilities  each  task  requires. 
For  example,  a  mechanic  can  become  aware  of  skills 
an  NDT  inspector  has  and  constraints  under  which  he 
or  she  works.  Role-playing  helps  each  member 
develop  a  better  understanding,  e.g.,  mental  model, 
of  each  task  and  of  interdependencies  between  and 
among  tasks.  With  role-playing,  trainees  have  the 
opportunity  to  experience  on-the-job  problems  and  to 
explore  specific  solutions  to  them  (Gordon,  1994). 

11.3.4  Task  Demonstration 

Task  demonstration  has  been  successfully  used  for 
team  training.  A  task  demonstration  assists  trainees 
by  showing  where  and  how  individual  team  members 
make  inputs  and  can  be  most  helpful  for  context- 
specific  skills  (Cannon-Bowers,  et  al.,  In  Press).  A 
passive  demonstration  could  be  a  computer 
simulation  of  a  task  or  an  illustration  consisting  of 
flow  diagrams.  A  passive  demonstration  helps 
trainees  identify  critical  task  elements;  determine 
how  each  team  member  contributes;  understand  the 
sequence  of  subtasks;  establish  step-by-step 
procedures;  and  identify  requirements  for 
coordination,  equipment  and  tooling.  For  aircraft 
maintenance,  when  computer  simulation  of  all  tasks 
is  not  feasible,  cross-training  is  possible  with 
simulations  of  representative  tasks  sharing  the  same 
critical  elements. 

11.3.5  Feedforward  Training 

Feedforward  training,  proven  effective  for 
individuals  (Drury  and  Gramopadhye,  1990), 
improves  performance  when  applied  to  teams 
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(Fredericksen  and  White,  1989).  Feedforward 
training  can  take  the  forms  of  physical  guidance, 
demonstrations,  or  verbal  advice.  It  advises  team 
members  about  upcoming  situations  so  that  they  are 
prepared.  For  example,  trainees  learn  how  a  team 
should  resolve  conflicts  arising  due  to  equipment 
being  unavailable,  or  how  to  respond  when 
instruction  procedures  ,  e.g.,  on  workcard,  are  not 
clear  and  are  ambiguous,  or  when  a  member  is 
assigned  a  different  task. 

11.3.6  Team  Decision-Making 

Team  decision-making  requires  educating  the  team 
on  how  to  utilize  various  pieces  of  information  to 
reach  an  optimal  decision  (Hogan,  et  al.,  1991).  The 
method  involves  training  members  on  decision¬ 
making  techniques,  ranging  from  decision  by 
consensus  to  brainstorming,  to  using  nominal  group 
techniques.  Not  all  these  techniques  apply  to  or  are 
relevant  for  training  AMT  teams.  The  team  decision¬ 
making  dimension  is  similar  to  communication 
because  teams  need  to  know  what,  why,  where  and 
how  information  can  be  accessed  for  optimal 
decisions  (Swezey  and  Salas,  1992). 

11.3.7  Feedback  Training 

Feedback  training,  i.e.,  knowledge  of  results,  is 
beneficial  for  individual  skills  training  (Patrick, 

1992;  Czaja  and  Drury,  1981),  and  a  similar  effect 
exists  for  teams  (Dyer,  1984;  Nieva,  et  al.,  1978).  In 
fact,  practice  without  feedback  degrades  a  team’s 
proficiency.  Cannon-Bowers  et  al  (In  Press)  write, 
"Feedback  improves  skill  acquisition  and  subsequent 
task  performance  by  reinforcing  learning,  by 
providing  cues  for  goal  setting  and  adjustment,  and 
by  reducing  the  negative  effects  of  self-serving 
attributions  and  social  loafing." 

The  following  factors  are  essential  for  providing 
effective  feedback: 

Timing:  Feedback  should  be  timely.  Team 
performance  is  generally  superior  when  feedback  is 
immediate,  rather  than  delayed. 

Focus:  Feedback’s  focus  is  important.  Providing 
feedback  on  only  certain  aspects  of  a  task  results  in 
performance  improvements  on  only  that  aspect  of  the 
task.  Team  training  should  not  emphasize  one  aspect 
of  team  performance  more  than  others. 

Sequence:  Initial  feedback  should  be  provided  on  one 
aspect  of  a  task;  later  feedback,  on  all  aspects  of  a 


task.  This  sequence  allows  trainees  to  focus  on  all 
aspects  of  team  tasks. 

Feedback  Mix:  The  ratio  of  individual  to  team 
feedback  also  effects  team  performance.  Individual 
feedback  should  be  provided  during  the  initial 
training  session  to  train  individuals  to  a  criterion 
level  of  performance.  Feedback  on  later  sessions 
should  address  team  aspects  of  performance.  This 
strategy  ensures  that  individual  skills  are  suitably 
developed  before  team  feedback  is  provided  while 
also  preventing  individual  members  from  developing 
misconceptions  about  their  own  performance  when 
the  team  receives  feedback. 


To  test  the  effectiveness  and  usefulness  of  team 
training  as  a  strategy  for  improving  team 
performance  for  aircraft  maintenance,  we  conducted 
a  study  with  AMTs  from  an  FAA-licensed  A  &  P 
school.  Current  analyses  are  based  on  the  hypothesis 
that  teams  successfully  completing  team  training 
exhibit  specific  interaction,  communication,  and 
coordination  behaviors  enhancing  their  performance. 
In  this  study,  we  addressed  the  following  questions: 

•  Does  team  training  effectively  improve  overall 
team  performance? 

•  Do  effective  and  less-effective  teams  display 
different  types  of  team  behaviors? 

•  Can  team  training  enhance 

interactive/communication  behaviors? 

We  designed  the  experiment  described  below  to  test 
the  hypothesis  and  to  answer  the  questions.  We  do 
not  provide  complete  details  below,  but  eventually 
will  publish  them  as  a  sequence  of  technical  papers. 

11.4.1  Subjects 

The  participants  in  this  study  were  24  male  students 
AMTs  between  20  and  30  years  old  from  an  FAA- 
licensed  A  &  P  school.  All  subjects  were  in  the 
second  year  of  a  two-year  curriculum. 

11.4.2  Task 

The  task  consisted  of  two  distinct  sessions:  the 
removal  and  the  installation  of  a  turbine  engine  from 
a  Beechcraft  airplane.  Major  phases  in  the  removal  of 
the  engine  are  external  preparation,  engine 
preparation,  and  engine  extraction.  Major  phases  in 
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engine  installation  are  engine  installation,  engine 
preparation,  and  external  preparation.  Details  of  each 
phase  are  outlined  in  Table  1 1.2  (Chapter  1 1  - 
Appendix).  We  selected  this  task  based  on  its  high 
potential  for  teamwork.  It  necessitates  more  than  one 
person  and  requires  a  significantly  high  degree  of 
coordination  and  communication  between  team 
members  for  its  successful  completion. 

11.4,3  Frocedere 

Each  subject  completed  a  demographics  form  (Table 
1 1.3,  Chapter  1 1  -  Appendix)  and  was  randomly 
assigned  to  one  of  eight  three-person  teams.  Four 
teams  served  as  the  control  group,  and  remaining 
four  teams  received  team  skills  training  (this  was 
team  training  group).  Initially,  all  subjects  in  the 
control  group  and  the  team  training  group  received 
individual  skills  training  that  provided  technical 
information  on  how  a  turbine  engine  works,  on  the 
theory  of  turbine  engines,  and  on  major  steps  for 
removing  and  installing  the  engine.  Subjects  also 
received  detailed  information  about  different  tools 
and  their  proper  uses;  tools  used  are  listed  in  the 
Chapter  1 1  -  Appendix  as  Table  1 1 .4,  After 
individual  skills  training,  teams  in  the  training  group 
received  team  training.  Before  starting  the  team 
training,  teams  in  the  training  group  performed  a 
warm-up  team  exercise  (see  Chapter  1 1  -  Appendix, 
Table  11.5). 

The  team  training  program  was  developed  in 
cooperation  with  trainers  and  key  personnel  of  a 
major  aircraft  repair  and  overhaul  facility  and 
instructors  from  an  A  &  P  school.  The  training 
program  used  some,  though  not  all,  of  the  team 
training  strategies  we  described  above.  We  combined 
the  team  skills  with  team  training  research  to  develop 
a  behaviorally  based,  team  training  program  focused 
on  improving  specific  team  skills.  First,  we  tested  the 
team  training  program  using  AMTs  from  our  partner 
repair  facility  for  a  specific  aircraft  maintenance  task. 
However,  we  do  not  report  results  of  the  field  study 
at  the  aircraft  repair  facility;  they  are  forthcoming  in 
other  papers.  We  modified  and  refined  our  team 
training  program  based  on  the  field  study’s  results 
and  used  the  revised  version  in  the  current  study.  The 
training  program  had  five  stages,  with  each  stage 
requiring  2-3  hours  (see  Chapter  1 1  -  Appendix, 
Table  1 1.6).  Teams  remained  intact  through  the 
entire  team  training  process  and  the  study’s  duration. 
Following  team  training,  teams  in  the  training  group 
performed  the  engine  removal  and  installation  task. 


Teams  in  the  control  group  performed  the  same  task. 
Unlike  the  team  training  groups,  control  group  teams 
performed  the  task  directly  after  they  received 
individual  skills  training.  When  they  completed  the 
entire  task,  we  debriefed  all  teams  and  thanked  them 
for  participating. 


11,5.1  Teamwork  Skills 


A  series  of  recent  studies  conducted  with  military 
teams  offer  insight  into  measuring  the  team 
process  (Morgan,  et  al.,  1986;  Baker  and  Salas, 

1992).  Studies  in  teamwork  assessment  show  that  it  is 
possible  to  observe  and  record  changes  in  team 
behavior  and  to  discriminate  more-effective  from 
less-effective  teams  (Oser,  et  al.,  1989).  Our  detailed 
review  of  teamwork  measurement  literature  suggests 
that  team  process  measures  rely  heavily  on 
observation  (Schiflett,  et  al.,  1985;  Morgan,  et  al., 
1986)  and  that  team  studies  use  behaviorally 
anchored  rating  scales  for  data  collection.  For  the 
current  study,  assessment  tools  (rating  scales)  were 
developed  and  refined  to  measure  teamwork  skills 
and  team  task  performance. 


We  collected  two  types  of  data  on  the  previously 
mentioned  team  skill  dimensions  by  interviewing 
team  members  and  instructors.  One  type  of  data 
reflected  instructors’  observations;  the  other,  team 
members’  perceptions.  We  collected  the  first  type  of 
data  with  the  instructors’  interviews  (Chapter  1 1  - 
Appendix,  Table  1 1.7).  We  collected  the  second  type 
with  the  post-session  interviews  (Chapter  1 1  - 
Appendix,  Table  1 1.8).  Both  the  interviews  use  a 
Likert-type,  seven  point,  agree-disagree  scale: 
trainees  and  instructors  indicated  their  response  to 
each  item.  Instructors  and  student  AMTs  completed 
the  respective  intervies  on  completion  of  each 
session,  i.e.,  engine  removal  and  engine  installation. 


11.5.2  Team  Attitude 

Attitude  measures  attempt  to  gauge  the  trainees’ 
opinions  about  whether  they  believe  that  training  and 
teamwork  will  improve  team  performance.  One  of 
the  most  popular  attitude  measurement 
questionnaires  is  the  CMAQ  (Cockpit  Management 
Attitudes  Questionairre)  for  assessing  commercial 
aviators'  attitudes  about  team  training  (Helmreich  et 
al.,  1986).  In  the  current  study,  we  used  a  modified 
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version  of  an  attitude  questionnaire  (Chapter  1 1  - 
Appendix,  Tables  11.9  and  11.10)  in  our  interviews, 
administering  it  to  student  AMTs  before  the  study’s 
commencement  and  after  its  completion. 

11.5.3  Task  Performance 

In  addition  to  data  on  team  behavior,  data  were  also 
collected  on  speed,  accuracy,  and  safety  measures. 
We  recorded  this  data  using  the  data  collection 
instrument  in  Chapter  1 1  -  Appendix,  Table  11.11. 
Data  were  collected  on  the  above-listed  task 
performance  measures  for  each  phase  of  the  engine 
removal  and  engine  installation  tasks.  Results  are 
reported  with  the  Task  Performance  Summary  Table 
(see  Chapter  11  -  Appendix,  Table  11.12). 


This  study’s  results  are  indicative  since 

comparisons  are  based  on  only  four  teams  per 
group  (training,  control).  However,  these  results  do 
generally  indicate  that  we  are  heading  in  the  right 
direction.  The  data  collection  instruments  and  task 
performance  summary  provided  data  for  24 
individuals  from  8  teams.  These  data  are  reported  in 
this  section,  divided  into  findings  based  on  data  from 
the  instructors’  evaluations,  from  self-evaluations, 
and  from  the  task  performance  summary. 

Figures  1 1.2  and  1 1.3  (Chapter  1 1  -  Appendix)  show 
instructors'  overall  ratings  for  the  trained  and 
untrained  teams  on  each  team  skill  dimension.  The 
instructor's  ratings  on  the  instructors’  interview  were 
mapped  onto  different  team  skills.  The  chart  shows 
that  teams  which  had  team  training  were  ranked 
equal  to  or  better  than  teams  which  did  not  have  team 
training  on  each  team  skill  dimension  for  both  engine 
removal  and  engine  installation  phases.  These  results 
suggest  that  teamwork  skills  of  the  teams  receiving 
training  were  perceived  to  be  much  better  than  those 
of  teams  not  receiving  training.  Since  no  data  were 
collected  on  individual  team  members,  it  is  not 
possible  to  assess  each  individual’s  relative 
performance. 

It  is  interesting  to  note  that  performance  differences 
between  trained  and  untrained  teams  are  much  larger 
on  the  engine  removal  phase  (first  session)  than  on 
the  engine  installation  phase  (second  session).  Teams 
which  did  not  receive  training  showed  improvement 
and  better  teamwork  in  the  latter  phase  (engine 
installation).  This  could  be  because  team  interaction 
patterns  are  established,  lessons  are  learned,  and 


communication  norms  develop  as  the  task  proceeds. 
Experience  helps  refine  the  team’s  interaction 
process  so  that  it  works  more  effectively  on 
subsequent  tasks.  Much  of  the  team  evolution  and 
maturation  process  for  teams  not  receiving  training 
was  completed  "on-the-job,"  while  a  large  portion  of 
this  process  for  trained  teams  was  completed  during 
training.  Despite  differences,  the  data  indicate  team 
evolution  and  maturation  effects  for  both  teams. 

These  results  add  weight  to  the  claim  that  effective 
team  behaviors  can  be  identified  and  enhanced  by 
having  teams  engage  in  those  behaviors  in  a  training 
environment. 

To  understand  individual  team  members’  perception 
of  their  team’s  performance,  we  analyzed  the  Post¬ 
session  self-evaluation  interview.  Results  are 
reported  in  the  Chapter  1 1  -  Appendix  as  Figures 
1 1.4  and  1 1.5.  Although  the  instructors’  analysis  of 
trained  and  untrained  teams  revealed  a  large 
difference  in  various  team  behaviors,  we  did  not  find 
a  similar  large  effect  here.  Nevertheless,  results  of  the 
self-evaluation  interview  are  that  the  trained  group’s 
mean  score  was  higher  than  the  control  group’s  on 
five  of  six  team  skills  measures  on  the  engine 
removal  task  and  on  four  of  six  measures  on  the 
engine  installation  task.  To  gauge  teams’  attitudes 
towards  teamwork  and  their  understanding  of  the 
principles  of  teamwork,  we  analyzed  pre-  and  post¬ 
training  interviews.  Figure  11.6  (Chapter  11  - 
Appendix)  shows  that,  although  scores  for  both  the 
trained  and  the  untrained  groups  are  comparable  on 
the  pre-training  interviews,  there  are  differences  on 
the  post-training  interviews.  The  trained  group’s 
higher  scores  on  six  of  eight  questions  reflect  the 
effect  of  training  in  and  understanding  of  teamwork 
and  team  principles. 

To  understand  whether  improved  team  performance 
translated  into  improved  task  performance,  we 
collected  task  performance  measures  for  both  groups. 
The  data  for  the  trained  and  control  groups  are 
summarized  in  Table  11.12  (Chapter  1 1  -  Appendix). 
Measure  1  relates  to  speed;  measures  2,  3  and  4,  to 
accuracy;  and  measures  5  and  6,  to  safety.  Teams  in 
the  untrained  (control)  group  required  significantly 
more  time  to  complete  the  engine  removal  task. 
However,  there  was  not  a  large  difference  on  the 
engine  installation  task.  This  result  could  be 
attributed  to  the  lack  of  coordination  and 
communication  among  members  of  the  control  group 
present  in  the  first  stage  and  absent  in  the  second. 
Over  time,  teams  in  the  control  group  improved 
coordination  and  communication,  resulting  in 
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reduced  task  time  on  the  engine  installation  task. 
Similarly,  the  trained  group  made  fewer  errors  for 
both  engine  removal  and  installation  tasks  and  had 
superior  scores  on  accuracy  measures  2,  3  and  4.  No 
significant  differences  were  observed  between  the 
groups  on  safety  measures.  The  most  important  result 
is  that  trained  teams  with  effective  team  behaviors 
were  overall  more  effective  and  more  efficient. 
Trained  teams  demonstrated  more  behavior  involving 
coordination  and  communication  skills,  i.e., 
coordinating  gathering  information,  conveying  the 
right  information  to  the  right  person  at  the  right  time 
in  the  right  format,  receiving  relevant  information; 
error-correction  skills,  i.e.,  providing  team  members 
with  performance  feedback  and  helping  resolve 
en'ors;  and  interpersonal  skills;  i.e.,  leadership, 
displaying  appreciation  for  help  provided,  and 
making  team-building  statements.  These  behaviors 
resulted  in  improved  task  performance. 

A  correlation  exists  between  successful  team 
behavior  and  task  performance.  Though  limited  in  its 
sample  size,  this  study’s  results  indicate  that  training 
AMTs  on  team  skills  improve  coordination  and 
communication  skills.  In  turn,  this  translates  into 
improved  task  performance. 


This  study  was  a  first  effort  devoted  expressly  to 
evaluating  the  effect  of  team  training  in  the 
aircraft  maintenance  environment.  The  study’s 
implications  are  encouraging  as  to  the  potential  team 
training  has  for  improving  team  performance  and 
overall  task  performance.  We  draw  the  following 
specific  conclusions  from  this  study; 

o  It  is  possible  to  identify  team  skills  and  to  train 
student  AMTs  in  teamwork  skills  critical  for 
successful  team  performance  in  the  aircraft 
maintenance  environment. 

®  Teams  which  receive  team  training  exhibit  a 
larger  percentage  of  behaviors  related  to  team 
performance.  Also,  results  suggest  that  members 
of  teams  which  did  not  receive  team  training  do 
not  exhibit  the  high  percentages  of  team 
behaviors  as  members  of  more-effective  teams. 

Based  on  this  study’s  results,  training  for  student 
AMTs  should  emphasize  generic  and  context-specific 
team  skills,  focusing  on  coordination, 
communication,  interpersonal,  and  leadership  skills. 
Our  findings  provide  insight  for  developing  future 


team  training  systems  and  for  improving  existing 
instructional  technology.  The  elements  of  the  team 
training  program  outlined  in  this  study  can  easily  be 
incorporated  into  A  &  P  school  curricula  to  prepare 
student  AMTs  for  teamwork.  Further,  elements  of  the 
team  training  program  can  also  be  incorporated  into 
formal  methodology  used  to  train  AMTs  at  different 
aircraft  sites.  The  operational  setting  for  the  current 
study  provided  the  opportunity  to  observe  teams  in 
the  field,  rather  than  in  a  laboratory.  Although  results 
are  encouraging,  additional  team  research  is  needed 
to  fully  understand  complex  interactions  existing  in  a 
team  environment  for  different  tasks  and  conditions. 
The  following  section  outlines  how  team  training  can 
be  incorporated  in  a  typical  A  &  P  school  curriculum 
and  provides  a  functional  description  of  a  computer- 
based  tool  for  team  training  which  will  be  developed 
under  Phase  YI  of  this  contract. 


The  previous  study  demonstrated  team  training’s 
effectiveness  for  improving  both  teamwork  skills 
and  task  performance  for  a  specific  aircraft 
maintenance  task,  using  student  AMTs.  The  results 
of  the  controlled  study  and  recognition  of  the 
important  role  of  teamwork  establish  a  need  to 
identify  team  projects  which  can  train  student  AMTs 
in  teamwork  skills  and  prepare  them  for  cooperative 
environments.  This  section  outlines  specific  team¬ 
training  projects  which  could  be  used  in  a  typical 
FAA-licensed  A  &  P  school  curriculum.  Table  11.13 
(Chapter  1 1  -  Appendix)  outlines  a  typical  A  &  P 
school  curriculum,  and  Chapter  1 1  -  Appendix,  Table 
1 1.14  presents  a  condensed  overview  of  various  team 
projects  which  could  be  incorporated  therein. 

11.8.1  Computer-Based  Tool  for  Team  Training 

As  computer-based  technology  becomes  increasingly 
cheaper,  the  future  will  see  an  increased  application 
of  advanced  technology  in  training.  Over  the  past 
decade,  instructional  technologists  have  provided 
numerous  technology-based  training  devices 
promising  improved  efficiency  and  effectiveness. 
Examples  include  computer  simulation,  interactive 
video  discs,  and  other  derivatives  of  computer-based 
applications  (Johnson,  1990(a)).  The  compact  disc 
read  only  memory  (CD-ROM)  and  digital  video 
interactive  (DVI)  are  examples  of  other  types  of 
technologies  which  will  provide  future  "multi-media” 
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training  systems.  Technologies  such  as  Computer- 
Aided  Instruction  (CAI),  Computer-Based  Training 
(CBT),  and  Intelligent  Tutoring  System  (ITS)  are 
being  used  today,  ushering  in  a  revolution  in  training. 
Several  new  technologies  have  found  a  place  in 
maintenance  training  (Johnson,  1990(a),  1992; 
Shepherd,  1992). 

Hypermedia  is  a  tool/instructional  system  finding 
acceptance  as  a  tool  for  learning  among  learning 
theorists.  Hypermedia  involves  non-linear 
organization  of  information,  linking  together  discrete 
blocks  (chunks)  of  information  to  create  an 
information  network.  It  can  also  be  seen  as  a  non¬ 
sequential  method  for  presenting  and  accessing 
information  in  which  users  can  move  freely 
according  to  their  needs.  Hypermedia  information  is 
multimedia:  text,  graphics,  animation,  and  audio.  If 
information  is  only  text,  it  is  known  as  hypertext. 
Hypermedia  systems  have  found  extensive  use  in 
applications  ranging  from  browsing  to  training. 
Jonassen  and  Gabringer  (1990)  list  examples  of 
hypermedia  in  instructional  tools  such  as  language 
learning,  science  teaching,  and  browsing  in 
encyclopedias.  Christensen,  et  al.  (1993)  developed  a 
hypermedia-based  instructional  tool  for  teaching 
hypermedia  system  design.  Koshy,  et  al.  (In  Press) 
developed  a  hypermedia  version  of  a  maintenance 
manual  for  diagnostic  training.  In  each  case, 
hypermedia  was  useful  for  learning  and  training 
applications. 

The  current  research  effort  was  devoted  expressly  to 
facilitating  understanding  and  to  examining  how 
team  members  interact  and  how  team  training  can 
facilitate  teamwork  in  the  aircraft  maintenance 
environment.  Having  met  these  goals,  our  next  step  is 
to  consider  training  media  which  uses  instructional 
techniques  developed  in  this  phase  of  the  research  in 
order  to  develop  a  training  program  enhancing  team 
skills.  Hypermedia  has  the  potential  to  enhance 
learning  and  could  prove  to  be  useful  for  improving 
certain  aspects  of  teamwork.  In  the  next  phase  of  our 
research,  we  propose  to  develop  a  hypermedia- based 
training  tool  designed  to  support  learning  teamwork 
in  the  aircraft  inspection  and  maintenance 
environment.  We  provide  a  functional  description  of 
the  proposed  training  tool  below. 

11.8.2  Functional  Description 

The  Aircraft  Maintenance  Team  Training  (AMTT) 
software  will  be  a  computer-based  hypermedia 
system  for  team  training.  It  will  be  developed  for 


student  AMTs,  focusing  on  generic  and  context- 
specific  team  skills.  The  system  will  be  programmed 
using  Visual  Basic/Tool  Book  to  operate  on  an  IBM- 
compatible  computer  (486  DX2/66  Hz  ,  8  Mb  of 
RAM),  using  Microsoft  Windows  and  utilizing 
multiple  media  such  as  sound,  text,  animation  and 
graphics.  AMTT  will  consist  of  the  two  basic 
modules  and  other  sub-modules  outlined  below. 

11.8.2.1  The  Trainee's  Module 

The  Trainee’s  Module  will  train  AMTs  on  various 
aspects  of  teamwork,  including  generic  and  context- 
specific  team  skills.  It  will  include  the  following 
basic  elements: 

11.82.1.1  Team  Overview  Module 
Introduction:  This  module  will  introduce  trainees  to 
the  basics  and  objectives  of  teamwork  (team 
mission).  This  module  will  use  the  Landing  on  the 
Moon  exercise  to  demonstrate  the  importance  of 
teamwork.  The  importance  of  and  need  for  teamwork 
in  aircraft  inspection  and  maintenance  will  also  be 
emphasized,  identifying  basic  team  skills  and 
illustrating  each  skill’s  importance. 

Tools  for  Making  Team  Decisions:  This  submodule 
will  introduce  trainees  to  decision-making 
techniques,  providing  examples  of  using  the 
techniques  in  the  aircraft  maintenance  environment. 

Team  Communication:  This  submodule  will 
introduce  trainees  to  aspects  of  written  and 
verbal/nonverbal  team  communication,  providing 
illustrations  of  appropriate  and  inappropriate 
communication  in  the  aircraft  maintenance 
environment.  Specifically,  communication  examples 
will  focus  on:  format,  direction,  frequency,  length, 
conditions,  context,  and  time.  The  importance  of 
good  communication  for  team  performance  will  be 
emphasized. 

Team  Feedback:  This  submodule  will  provide 
trainees  with  guidelines  for  providing,  receiving,  and 
using  feedback  to  communicate  with  other  AMTs 
clearly  about  how  tasks  are  being  performed. 

Team  Coordination:  This  submodule  will  focus  on 
the  coordination  required  for  team  members  to  ensure 
well-orchestrated  teamwork. 

Team  Leadership:  This  submodule  will  focus  on  the 
critical  role  of  team  leadership  for  accomplishing 
team  tasks.  For  example,  team  members  will  be 


199 


Team  Training 


Chapter  1 1 


shown  how  to  handle  information  overload  under 
stressful  conditions,  specific  behaviors  exhibiting 
leadership  and  assertiveness,  and  methods  of 
motivating  others. 

Team  Evaluation:  This  submodule  will  expose  the 
trainees  to  the  instruments  used  to  evaluate  individual 
and  team  performance  on  a  task. 

Each  submodule  will  first  introduce  trainees  to  basic 
principles  and  the  provides  examples  applying  the 
principles  to  enhance  teamwork  in  the  aircraft 
maintenance  environment.  Trainees  will  make  an 
active  response  as  they  are  exposed  to  new  material 
and  will  be  provided  with  immediate  feedback  as  to 
their  answer’s  correctness.  This  stage  will  be 
followed  by  a  question  and  answer  session  for  the 
material. 

1L8.2.L2  Team  Building  Exercise  Module 
This  module’s  objective  is  to  demonstrate  the 
application  of  basic  principles  of  teamwork 
emphasized  in  the  Team  Overview  Module.  Trainees 
will  undertake  a  series  of  exercises  requiring  them  to 
demonstrate  their  understanding  of  principles.  The 
training  will  use  training  strategies  such  as  role- 
playing,  feedforward,  and  feedback.  For  example, 
roles  of  various  team  members  will  be  modeled  for 
certain  task  situations,  using  knowledge  from  experts. 
Examples  of  how  interactions  could  proceed,  with 
examples  of  poor  and  good  behavior,  will  be 
demonstrated  via  simulation.  Trainees  will  comment 
on  the  behavior’s  appropriateness  and  will  be  asked 
for  inputs  or  suggestions  to  improve  team 
performance.  Trainees  will  be  given  guidance  and 
feedback  during  and  after  the  session. 

1L8.2J3  Task  Simulation  Module 
This  module  will  provide  trainees  with  graphical 
demonstration,  animation,  and  flow  charts  of 
different  scenarios  for  select  aircraft  maintenance 
tasks.  Team  members  using  this  module  can  interact 
cooperatively  to  identify  ways  to  improve  teamwork 
for  the  representative  simulated  aircraft  maintenance 
tasks. 

1 1 .8.2.2  The  Instructors  Module 

1L8.2.2.1  Assessment  Module 
This  module  will  provide  the  instructors  with  a 
means  to  assess  trainees’  understanding  of  using  team 
principles  and  will  allow  instructors  to  evaluate 
trainee’s  and  the  team’s  performance  while 


interacting  with  AMTT  software.  The  module  will 
provide  the  instructor  with  various  data  collection 
instruments  used  by  both  trainees  and  instructors. 

11.8.2.2.2  Report  Generation  Module 
The  Report  Generation  Module  will  allow  instructors 
to  print  reports  of  results.  It  will  also  allow 
instructors  to  generate  printouts  of  data  collection 
instruments  and  select  material  in  the  Team 
Overview  Module.  This  will  allow  instructors  to  use 
the  material  in  a  classroom  environment  and  to  use 
data  collection  instruments  for  field  study. 
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THE  INPUT 

EXTERNAL  FACTORS 

ORGANIZATIONAL  FACTORS 

-  Size  of  organization 

-  type  (e.g.,  airline,  generai  aviation, 
repair  faciirty) 

-  management  structure 

-  quaiity  circles 

-  organizational  climate 

-  communication  norms 


THE  PROCESS 


THE  OUTPUT 


ENVIRONMENTAL  FACTORS 

-  level  of  environmental  stress  (hangar, 
flightiine) 

*  environmental  uncertainty 

EQUIPMENT  FACTORS 

-  automation 

-  complexity 

-  SjDecializotion 

-  equipment  availability 

-  safety 

TASK  FACTORS 

-  type  of  check  (A,  B,  C  or  Heavy 
Check) 

-  task  type  (e.g.,  avionics,  power  plant, 
hydraulics,  sheet  metal  frame) 

-  task  complexity 

-  task  structure  (e.g.,  procedural, 
diagnostic  combination) 


>  TEAM  PROCESS 
OVER  TIME 


INTERNAL  FACTORS  | 

INDMDUAL  FACTORS 

TEAM  FACTORS 

can  be  represented 

can  be 

by: 

represented  by: 

-  knowledge 

-  interpersonal  skills 

-skills 

-composition  of 

-abilities 

the  team  i 

-  mental  models 

-number  of 
members  In  the  I 

team  1 

-length  of  time  the  | 

team  members  j 

hove  been  | 

working  together  | 

▲  Attitude  A 

TRAINING 


INDIVIDUAL  TRAINING 
can  be: 

-  active  training 

-  schema  training 

-  feedforward 
training 

-  progressive  part 
training 

-  adaptive  training 

-  cognitive  feedback 


TEAM  TRAINING 
can  be: 
-guidance 

-  task  simulation 

-  demonstration/ 
lecturing 

-  role  playing 
-task 

walkthrough 

-  part/whole  task 
training 


PROCESS  MEASURES 
IndMdual  Process  Measures 
can  be  measured  by  changes  in  the 
following  for  each  member: 

-  knowledge 

-  skills 
-abilities 

-  mental  models 


J  Team  Process  Measures 
I  can  be  measured  by  changes  in: 

I  -  team  skills  (ability  to  co-ordinate, 
communicate.  Interpersonal  skills) 

I  -  development  of  new  team  processes 

1  Task  Performance  Measures 
j  can  be  measured  in  temns  of  the 
5  following  for  the  overall  task  and  each 
j  sub-task: 
j  -  speed 
j  -  accuracy 

J  -  safety  (i.e.,  endangering  safety  of  self 
and  other  team  members  while 
performing  the  task). 


Feedback 
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Team  Skills 

Description 

1 .  Coordination 

This  refers  to  the  team’s  ability  to  organize  available  resources 
and  activities  so  as  to  accomplish  the  goal  within  the  temporal 
constraints. 

2.  Communication 

The  process  by  which  the  team  members  clearly  and 
accurately  exchange  information,  using  established  procedures 
and  language.  It  also  encompasses  the  team  members’  ability 
to  receive  and  provide  constructive  feedback  on  the 
performance  of  other  team  member{s)  so  as  to  help  achieve 
the  team  goal. 

3.  Cohesiveness 

This  refers  to  the  process  by  which  all  members  of  the  team 
develop  compatible  models  of  the  system  and  work  together  as 
one  unit. 

4.  Decision-Making 

This  refers  to  the  process  by  which  teams  can  use  judgement, 
analytical  technique,  and  consensus  methods  to  arrive  at 
decisions  by  pooling  together  information  and  resources. 

5.  Interpersonal 

This  refers  to  team  members’  abilities  to  employ  cooperative 
behavior  to  resolve  interpersonal  problems  and  optimize 
member  interactions. 

6.  Leadership 

This  refers  to  the  ability  to  assign,  plan,  organize,  and  motivate 
members  to  accomplish  the  goal. 

7.  Attitude 
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ENGIME  REMOVAL 

ENGINE  INSTALLATION 

1 .  External  Preparation 

a)  Set  up  tail  stand 

b)  Disconnect  electric  power 

c)  Remove  top  cowling 

d)  Disconnect  actuator 

e)  Remove  bottom  cowling 

1.  Engine  Installation 

a)  Install  Engine 

b)  Connect  top  V-brace 

c)  Connect  lower  engine  mounts 

d)  Put  bulhead  bolts 

e)  Unmount  sling  and  hoist  from 
engine 

2.  Engine  Preparation 

a)  Remove  hoses  and  fittings 

b)  Disconnect  electrical  leads 

c)  Disconnect  engine  controls 

d)  Drain  oil 

e)  Remove  propeller 

2.  Engine  Preparation 

a)  Install  propeller 

b)  Fill  oil 

c)  Connect  engine  controls 

d)  Connect  electrical  leads 

e)  Put  back  hoses  and  fittings 

3.  Engine  Extraction 

a)  Mount  sling  and  hoist  on 
engine 

b)  Remove  bulkhead  bolts 

c)  Disconnect  lower  engine 
mounts 

d)  Disconnect  top  V-brace 

e)  Extract  Engine 

3.  External  Preparation 

a)  Put  back  bottom  cowling 

b)  Connect  actuator 

c)  Put  back  top  cowling 

d)  Connect  electric  power 

e)  Remove  tail  stand 

Chapter  1 1 


Appendix  -  Team  Training 


The  following  information  will  remain  confidential  and  is  for  research  purposes  only.  Each  team  member 
should  fill  in  all  questions  carefully  and  completely. 


DEMOGRAPHICS  FORM 


1.  Have  you  attended  a  technical  or  vocational  school  other  than  this  school? 

Yes _  No _ 

2.  If  you  answered  yes  to  question  1,  what  type  of  technical  training  did  you  receive? 


3.  Have  you  ever  worked  in  a  team  environment  prior  to  this  class? 

Yes _  No _  Not  Sure _ 

4.  If  you  answered  yes  to  question  3,  where  did  you  work  as  a  team  member? 

School _ Work _  Other _ 

4  (a).  What  kind  of  work  were  you  involved  in  as  a  team  member? 


5.  Have  you  ever  been  fully  employed  prior  to  attending  this  school? 

Yes _  No _ _ 

6.  What  kind  of  work  did  you  do? 


7.  Have  you  ever  had  any  team  training  before? 

Yes _  No _ 

8.  What  skills  did  you  learn? 


9.  Sex:  Male 

Female 

10.  Age:  17-20 

21-30 

31-40 

41-50 

51-60 

61  + 
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STUDENT  TOOL  LIST  -  REQUIRED  TOOLS 


1.  Tool  box 

6.  Socket  Set  1/4"  Drive 

No  larger  than  20  inches  high  X  20  inches  long.  (No 

5/32"  Regular  6  pt 

Rollaways) 

3/16" 

7/32" 

2.  Chain  and  lock 

1/4" 

9/32" 

3.  Open-End  Wrenches 

5/16" 

1/4  X  5/16" 

1 1/32" 

3/8  X  7/16" 

3/8" 

1/2  X  9/16" 

7/16" 

9/16  X  5/8" 

1/2" 

5/8  X  3/4" 

1/4"  Deep  6  pt 

11/16  X  13/16" 

5/16" 

3/4  X  7/8" 

3/8" 

15/16  X  1" 

7/16" 

1/2" 

4.  Box-End  Wrenches 

Ratchet 

1/4  X  5/16" 

Spinner  Handle 

3/8  X  7/16" 

Ext.  1  1/2" 

1/2  X  9/16" 

Ext.  3" 

9/16  X  5/8" 

Universal  Joint 

11/16  X  13/16" 

3/4  X  7/8" 

7.  Screw  Drivers 

15/16  X  1" 

Set  of  ten  - 

Range  of  Slotted  and  Phillips 

5.  Socket  Set  3/8"  Drive 

with  a  stubby  of  each. 

3/8"  Regular  12  pt 

7/16" 

8.  Punch 

1/2"  Deep  6  pt 

Pin  Punch 

9/16" 

1/16-1/8" 

5/8" 

Center  Punch 

11/16" 

3/8" 

3/4" 

Prick  Punch 

13/16" 

3/8- 

15/8"  Plug 

Line-Up  Tools 

Ratchet 

3/16x  9  &  5/32x7" 

3"  Ext. 

Case 

9.  Allen  Wrenches 

6"  Ext. 

Long 

7/8"  Deep  Spark  Plug  Socket 

5/64" 

Universal  Joint 

3/32" 

7/64" 

1/8" 

9/64” 

5/32" 

3/16" 

7/32” 

1/4" 
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10.  Adjustable  Wrenches  -  10" 

OPTIONAL  TOOLS 

11.  Measuring  Tape  12  ft. 

12.  Hammer,  Ball  Been  8  oz. 

1.  Cold  Chisels 

1/4,  3/8,  1/2,  3/4" 

2.  Allen  Wrenches 

13.  Hammer,  Plastic  Tip 

Short 

.050" 

14.  Flash  Light  2  Cell 

1/16" 

5/64" 

15.  Pliers,  Common  8" 

3/32" 

7/64" 

16,  Pliers,  Diagonal  T 

9/64" 

17.  Pliers,  Longnose  8" 

3.  Adjustable  Wrenches  -  6" 

18.  Pliers,  Duckbill 

4.  Machinist  Square 

19.  Pocket  Knife  4" 

5.  Hacksaw 

20.  Sheet  Metal  Snips 

Left 

6.  Hacksaw  blades 

Right 

7.  Pliers  Arc  Joint  9” 

21.  lOX  Magnifying  Glass 

8.  Socket  Set  1/2"  Drive 

Socket  Regular  12  pt 

22.  File  Set  -  8"  or  larger 

7/16" 

1  -  Bastard 

1/2" 

1  ’  Round 

9/16" 

1  -  Half  Round 

5/8" 

1  -  Triangular 

3/4" 

11/16" 

23.  File  Handles 

13/16" 

7/8" 

24.  File  Card 

15/16" 

1" 

Deep  12  pt 

1/2"  Ext.  6" 

25.  1  -  Extension  Type  Inspector  Mirror 

26.  1  “  Mechanical  Finger,  10-14  inch 

9/16"  Ext.  3" 

5/8" 

27.  1  -  Retrieving  Magnet,  10  -  14  inch 

3/4" 

11/16" 

28.  1  -  Thickness  Gage  Set 

13/16" 

.002  -  .035  or  better 

7/8" 

15/16" 

Ratchet 

Universal  Joint 
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Lost  On  The  Moon  Exercise 

Your  spaceship  has  just  crashed-landed  on  the  moon.  You  were  scheduled  to  rendezvous  with  a 
mother  ship  200  miles  away  on  the  lighted  surface  of  the  moon,  but  the  rough  landing  has  ruined  your  ship 
and  destroyed  all  the  equipment  on  board,  except  for  the  1 5  items  listed  below. 

Your  crew's  survival  depends  on  reaching  the  mother  ship,  so  you  must  choose  the  items  based  on 
their  importance  for  survival.  Place  number  one  by  the  most  important  for  survival.  Place  two  by  the 
second  most  important,  and  so  on  through  number  1 5,  the  least  important.  _ 


Your 
Ranking 
1  to  15 


NASA’s  Team  Team  Error 

Ranking  Ranking  Score 

ltol5  ItolS 


Box  of  matches 


Food  concentrate 


Fifty  feet  of  nylon  rope 


Parachute  silk 


Solar-powered  portable 
heating  unit 


Two  .45  caliber  pistols 


One  case  of  dehydrated  milk 


Two  1 00  pounds  tanks  of  oxygen 


Stellar  map  (of  the  moon's 
constellations) 


Self-inflating  life  raft 


Magnetic  compass 
Five  gallons  of  water 


Signal  flares 

First-aid  kit  containing  injection  needles 

Solar-powered  FM  receiver  - 
transmitter 

YOUR  TOTAL  ERROR  SCORE 
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Lost  On  The  Moon  -  Team  Rules 

1 .  Avoid  arguing  for  your  own  ranking.  Present  your  position  as  lucidly  and  logically  as  possible,  but 
listen  to  the  other  members'  reactions  and  consider  them  carefully  before  you  press  your  point. 

2.  Do  not  assume  that  someone  must  win  and  someone  must  lose  when  discussion  reaches  a  stalemate. 
Instead,  look  for  the  next-most-acceptable  alternative  for  all  parties. 

3.  Do  not  change  your  mind  simply  to  avoid  conflict  and  to  reach  agreement  and  harmony.  When 
agreement  seems  to  come  too  quickly  and  easily,  be  suspicious.  Explore  the  reasons  and  be  sure  everyone 
accepts  the  solution  for  basically  similar  or  complementary  reasons. 

4.  Avoid  conflict-reducing  techniques  such  as  majority  vote,  averages,  coin-flips  and  bargaining.  When 
a  dissenting  member  finally  agrees,  don't  feel  that  he  or  she  must  be  rewarded  by  having  his  or  her  own  way 
on  some  later  point. 

5.  Differences  of  opinion  are  natural  and  expected.  Seek  them  out  and  try  to  involve  everyone  in  the 

decision  process.  Disagreements  can  help  the  group's  decision  because  with  a  wide  range  of  information  and 
opinions,  there  is  a  greater  chance  that  the  group  will  hit  upon  more  adequate  solutions. _ 


Lost  On  the  Moon  -  Scoring 


Team  I  Team  2  Team  3  Team  4 

Total  Error  Points  _  _  _  _ 


Error  points  are  absolute  difference  between  your  rank  and  NASA's  (disregarding  plus  or  minus  signs) 

0  -  25  execellent 
26  -  32  good 
33  -  45  average 
46-55  fair 
56  -  70  poor 

71  -  112  very  poor  (suggest  possible  faking  or  use  of  earth  bound  logic) 
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■fsMfe  tilled 

Item 

NASA's  Reasoning 

NASA 

Rank 

Box  of  matches 

No  Oxygen  on  moon  to 
sustain  flame:  worthless 

15 

Food  concentrate 

Efficient  means  of 
supplying  energy 
requirements 

4 

Fifty  feet  of  nylon  rope 

Useful  in  scaling  cliffs, 
tying  injured  together 

6 

Parachute  silk 

Protection  from  sun's  rays 

8 

Solar-powered  portable 
heating  units 

Not  needed  unless  on  dark 
side 

Two  .45  caliber  pistols 

Possible  means  of  self 
propulsion 

11 

One  case  of  dehydrated 
Pet  milk 

Bulkier  duplication  of  food 
concentrate 

Two  100  pound  tanks 
of  oxygen 

Most  pressing  survival 
need 

1 

Stellar  map  (of  the 
moon's  constellations) 

Primary  means  of 
navigation 

3 

Self-inflating  life  raft 

C02  bottle  in  military  raft 
may  be  used  for  propulsion 

9 

Magnetic  compass 

Magnetic  field  on  moon  is 
not  polarized:  worthless 

14 

Five  gallons  of  water 

Replacement  for 
tremendous  liquid  loss  on 
lighted  side 

2 

Signal  flares 

Distress  signal  when 
mother  ship  is  sighted 

10 

First-aid  kit  containing 
injection  needles 

Needles  for  vitamins, 
medicines,  etc.  Will  fit 
aperture  in  NASA  space 
suit 

7 

Solar-powered  FM 
receiver  transmitter 

For  communication  with 
mother  ship,  but  requires 
line  of  sight  (short  range) 

5 

vr  > 
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Session  1  -  Basics  of  Teamwork 


Goals 

Provide  trainees  an  understanding  of  teams,  need  for  teamwork,  introduction  to  team  concepts, 
and  an  outline  of  future  sessions 

Major  Elements 

•  Initial  attitude  survey 

•  Why  there  is  a  need  for  teams 

•  Establish  the  need  for  consistency  and  clarity  in  goals:  team  goals  and  individual  goals 

•  Goals  of  team  building 

•  Team  work  exercise 

•  Overview  of  future  sessions 

Session  2:  Decision  Making 


Goals 

Introduce  trainees  to  scientific  approach  to  decision-making 

Major  Elements 

•  expose  trainees  to  different  tools  for  decision-making 

•  identify  the  merits  and  demerits  of  the  tools 

•  use  of  decision-making  tools  within  the  aircraft/maintenance  environment  context 
(which  tool?  when  to  use?  How  to  use?) 

•  exercise  involving  different  tools 

•  decision-making  by  consensus 

Session  3:  Group  Dynamics  1 :  Communication  and  Interpersonal 

Goals 

To  provide  each  trainee  with  an  understanding  of  the  essential  elements  of  communication 

Identify  steps  to  minimize  interpersonal  problems 

Major  elements 

•  establish  need  for  oral  communication  and  written  communication 

•  principles  of  good  communication  (format,  terminology,  direction,  when,  how,  how 
much/little) 

•  examples  of  appropriate  forms  of  communications  (written  and  oral)  within  the  aircraft 
maintenance  environment 

•  importance  of  providing  team  members  with  positive  and  negative  feedback  and  how 
to  receive  feedback  (When  to  give?  How  it  works?  How  to  receive?  ...) 

•  exercise  involving  correct  and  incorrect  communication  within  the  aircraft  maintenance 
environment 
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Session  4;  Group  Dynamics  2;  Coordination  and  Cohesiveness 
Goals 

To  train  on  the  importance  of  coordination  and  cohesiveness  in  achieving  the  team  goal 
Major  Elements 

9  Methods  to  eliminate  barriers  and  behavioral  problems 

9  Demonstrate  the  importance  of  coordination  as  it  relates  to  aircraft  maintenance  and 
inspection 

e  provide  examples  of  good  and  bad  coordination  and  demonstrate  the  effects  on 
task  performance 

9  Identify  every  member's  role  and  explain  interdependency 
9  Help  establish  accurate  expectations  of  the  contributions  of  other  team  members  to 
overall  performance 

Session  5:  Team  Activity 

Goals 

To  demonstrate  how  team  skills  can  improve  team  performance  for  an  aircraft 
inspection/maintenance  task 

Major  Elements 

9  construct  examples  of  team  activity 

9  Illustrate  importance  of  different  team  skills  in  accomplishing  the  activity 
9  role  play 

•  provide  feedback  to  teams 
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The  purpose  of  this  questionnaire  is  to  evaluate  the  effectiveness  of  team  training  on  team 
performance.  The  facilitator  is  in  a  position  to  observe  any  improvements  or  lack  of  improvements  in  team 
performance,  so  please  take  time  to  consider  each  statement.  All  responses  will  be  kept  confidential. 

Rate  each  statement  on  a  scale  of  1  -  7 


Number  of  times, 
Lowest,  Poor, 

Never,  etc. 

1  2 


Neutral 

4 


Highest,  Best,  Always, 
Very,  etc. 

7 


Team  1 

Team  2 

Team  3 

Team  4 

1 .  The  team  members  worked  well  together. 


2.  The  team  resolved  conflicts  effectively. 


3.  All  members  of  the  team  participated  in  the  decision¬ 
making  process 


4.  The  team  members  discussed  new  ways  to  tackle  the  task. 


5.  The  team  was  effective  in  establishing  ground  rules. 


6.  One  person  dominated  the  team. 


7.  There  was  at  least  one  person  who  was  disruptive. 


8.  There  was  at  least  one  person  who  did  not  participate  in 
team  discussions. 


9.  One  member  took  charge  of  assigning  the  tasks  and 
coordinating  activities  of  other  team  members. 


10.  Team  members  provided  each  other  with  performance 
fedback 

11.  The  team  members  worked  cohesively. 


12.  Team  members  responded  well  to  team  training. 


13.  The  team  members  follow  the  agenda 
(accomplished  the  objectives). 

14.  There  was  a  noticeable  improvement  due  to  team 
training. 
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Please  rate  the  following  statements  on  a  scale  of  1  -  7  by 

circling  the  response  that  best  fits  your  opinion 

concerning  the  statement.  All  response  will  be  kept  confidential. 

Definitely  Not 

1 .  The  team  followed  the  agenda  for 

the  session.  1  2  3 

4 

5 

6 

Definitely 

7 

2.  You  were  satisfied  with  the  level  of 
participation  by  team  members. 

1  2 

3 

4 

5 

6 

7 

3.  Everyone  contributed  and  was 
involved  in  team  decisions. 

1  2 

3 

4 

5 

6 

7 

4.  You  had  a  good  attitude  about  your 
work  and  the  task. 

1  2 

3 

4 

5 

6 

7 

5.  Team  members  allowed  personality 
conflicts  to  interfere  with  work. 

1  2 

3 

4 

5 

6 

7 

6.  You  were  satisfied  with  the  level  of  the 
teams'  achievement  towards  the 
established  goal. 

1  2 

3 

4 

5 

6 

7 

7.  Team  members  were  able  to  settle 
conflicts  effectively  among  themselves 

1  2 

3 

4 

5 

6 

7 

8.  You  feel  the  teams’  performance  was 
very  good. 

1  2 

3 

4 

5 

6 

7 

9.  You  feel  the  final  result  of  the  task 
was  very  good. 

1  2 

3 

4 

5 

6 

7 

10.  Your  opinion  was  considered. 

1  2 

3 

4 

5 

6 

7 

11.  One  member  took  charge  of  assigning 
the  tasks  and  coordinating  the  activities 
of  other  team  members. 

1  2 

3 

4 

5 

6 

7 

12.  Team  members  were  aware  of  each  others 
responsibilities. 

1  2 

3 

4 

5 

6 

7 

13.  You  were  satisfied  with  the  material 
used  for  team  training. 

1  2 

3 

4 

5 

6 

7 

14.  You  were  satisfied  with  the  material 
used  for  technical  training. 

1  2 

3 

4 

5 

6 

7 

15.  If  provided  with  another  opportunity, 
you  would  want  to  participate  in  a 
team  activity. 

1  2 

3 

4 

5 

6 

7 

16.  If  provided  with  another  opportunity, 
you  would  participate  in  a  team  activity 
with  the  same  group. 

1  2 

3 

4 

5 

6 

7 
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Please  circle  the  response  that  best  reflects  your  opinion  of  each  statement.  All  responses  will  be  kept  confidential. 


1 .  I  believe  teamwork  is  the 
best  way  to  accomplish 
work  tasks  in  all  situations. 

Strongly 

Disagree 

1  2 

3 

Neutral 

4 

5 

6 

Strongly 

Agree 

7 

2.  In  team  environments,  it  is 
important  to  follow  an  agenda. 

1 

2 

3 

4 

5 

6 

7 

3.  All  team  members  should 
contribute  to  team  decisions. 

1 

2 

3 

4 

5 

6 

7 

4.  If  one  team  member  doesn't 
understand,  other  team  members 
should  help  him  or  her.  1 

2 

3 

4 

5 

6 

7 

5.  Team  leaders  should  keep  the  team 
on  track  to  accomplish  goals. 

1 

2 

3 

4 

5 

6 

7 

6.  Team  decisions  are  superior  to 
individual  decisions. 

1 

2 

3 

4 

5 

6 

7 

7.  All  tasks  are  not  suited  for  team 
environments. 

1 

2 

3 

4 

5 

6 

7 

8.  I  am  comfortable  participating 
in  team  decisions. 

1 

2 

3 

4 

5 

6 

7 

9.  The  success  of  the  team  is  important 
to  each  individual. 

1 

2 

3 

4 

5 

6 

7 

10.  Training  improves  team 

performance.  1 

2 

3 

4 

5 

6 

7 
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Please  circle  the  response  that  best  reflects  your  opinion  of  each  statement.  All  responses  will  be  kept  confidential. 

Strongly  Strongly 

Disagree  Neutral  Agree 

1.  I  believe  teamwork  is  the 
best  way  to  accomplish 

work  tasks  in  all  situations. 

2.  In  team  environments,  it  is 

1  2 

3 

4 

5 

6 

7 

important  to  follow  an  agenda. 

3.  All  team  members  should 

1  2 

3 

4 

5 

6 

7 

contribute  to  team  decisions. 

4,  If  one  team  member  doesn't 
understand,  other  team  members 

1  2 

3 

4 

5 

6 

7 

should  help  him  or  her.  1 

5.  Team  leaders  should  keep  the  team 

2  3 

4 

5 

6 

7 

on  track  to  accomplish  goals. 

6.  Team  decisions  are  superior  to 

1  2 

3 

4 

5 

6 

7 

individual  decisions. 

7.  All  tasks  are  not  suited  for  team 

1  2 

3 

4 

5 

6 

7 

environments. 

8.  lam  comfortable  participating 

1  2 

3 

4 

5 

6 

7 

in  team  decisions. 

9.  The  success  of  the  team  is  important 

1  2 

3 

4 

5 

6 

7 

to  each  individual. 

10.  Training  improves  team 

1  2 

3 

4 

5 

6 

7 

performance.  1 

2  3 

4 

5 

6 

7 
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1 .  Total  time  to  complete  the  entire  task. 


1  (a).  Total  time  to  complete  the  External  Preparation  Phase. 


1  (b).  Total  time  to  complete  the  Engine  Preparation  Phase. 


1  (c).  Total  time  to  complete  the  Engine  Extraction  Phase. 


2.  Total  number  of  mistakes  made  by  the  team  while 
completing  the  entire  task. 


2  (a).  Total  number  of  mistakes  made  by  the  team  during 
External  Preparation  Phase. 


2  (b).  Total  number  of  mistakes  made  by  the  team  during 
Engine  Preparation  Phase. 


2  (c).  Total  number  of  mistakes  made  by  the  team  during 
Engine  Extraction  Phase. 


3.  Number  of  times  the  instructor  had  to  point  out  the 

mistakes  being  made  and  correct  them  during  the  entire  task. 


3  (a).  Number  of  times  the  instructor  had  to  point  out  the 

mistakes  being  made  and  correct  them  during  the  External 
Preparation  Phase. 


3  (b).  Number  of  times  the  instructor  had  to  point  out  the 

mistakes  being  made  and  correct  them  during  the  Engine 
Preparation  Phase. 


3  (c).  Number  of  times  the  instructor  had  to  point  out  the 

mistakes  being  made  and  correct  them  during  the  Engine 
Extraction  Phase. 

4.  Number  of  times  team  did  not  follow  correct  procedures 
during  the  entire  task. 


4  (a).  Number  of  times  team  did  not  follow  correct  procedures 
during  the  External  Preparation  Phase. 


4  (b).  Number  of  times  team  did  not  follow  correct  procedures 
during  the  Engine  Preparation  Phase. 


4  (c).  Number  of  times  team  did  not  follow  correct  procedures 
during  the  Engine  Extraction  Phase. 
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Engine  Removal  (averaged  over  4  teams)  - , 

Task  Performance  Measures 

Training  Group 

Control  Group 

1 .  Total  time  taken  to  complete  the  task  of  engine 
removal  (hrs./mins.) 

6  hrs  10  mins. 

7hrs  38  mins. 

2.  Number  of  mistakes  made  by  the  team  during  engine 
removal 

3 

9 

3.  Number  of  times  the  instructor  had  to  point  out  the 
mistakes  being  made  and  correct  them  during  the  task  of 
engine  removal 

3 

6 

4.  Number  of  times  the  team  did  not  follow  correct 
procedures  during  the  task  of  engine  removal 

1 

5 

5.  Number  of  times  safety  of  fellow  team  members  was 
endangered  during  the  task  of  engine  removal 

0 

0 

6.  Number  of  times  safety  procedures  were  not 
followed  during  the  task  of  engine  removal 

3 

1 

lliauaiiauwii  i 

Task  Performance  Measures 

Training  Group 

Control  Group 

1 .  Total  time  taken  to  complete  the  task  of  engine 
installation  (hrs./mins.) 

1 3  hrs  32  mins. 

14  hrs  15  mins 

2.  Total  number  of  mistakes  made  by  the  team  during 
engine  installation 

1 

5 

3.  Number  of  times  the  instructor  had  to  point  out  the 
mistakes  being  made  and  correct  them  during  the  task  of 
engine  installation 

2 

6 

4.  Number  of  times  the  team  did  not  follow  correct 
procedures  during  the  task  of  engine  installation 

2 

4 

5.  Number  of  times  safety  of  fellow  team  members  was 
endangered  during  the  task  of  engine  installation 

0 

0 

6.  Number  of  times  safety  procedures  were  not 
followed  during  the  task  of  engine  installation 

1 

2 
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Year  1 

Year  2 

Fall  Semester 

Fall  Semester 

General  Regulations 

Bonded  Structures  St  Welding 

Aircraft  Drawings 

Utlity  fit  Warning  Systems 

Ground  Handling  and  Servicing 

Landing  Gear  Systems 

Materials  and  Corrosion  Controi 

Airframe  inspection 

Assembly  and  Rigging 

Propellers  and  Components 

Algebra,  Geometry,  and  Trigonometry  1 

Professional  Communications 

Spring  Semester 

Spring  Semester 

Basic  Aircraft  Electricty 

Lubricating  Systems 

Wood,  Dope,  Fabric,  and  Finishes 

Ignition  Systems 

Sheet  Metal  Layout  and  Repair 

Turbine  Engine  Overhaul 

Reciprocating  Engine  Overhaul 

Engine  Inspection 

Conceptual  Physics  1 

Engine  Electrical,  Instrument,  and 

Fire  Protection  Elective 

Summer 

Aircraft  Environmental  Systems 

Summer 

Hydraulics  and  Pneumatic  Systems 

Powerplant  Fuel  Systems 

Aircraft  Electric  Systems 

Induction  Cooling  and  Exhuast 

Aircraft  Fuel  Systems 

Technology  and  Culture  Elective 

Personal/Interpersonal  Psychology 
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Year  1 

Course:  Ground  Handling  and  Services 

Team  project  title:  Aircraft  towing 
Number  of  team  members:  4 

Description:  Given  an  aircraft  and  aircraft  towing  equipment,  the  team  will  tow  aircraft  from  the  hangar  to  a 
preselected  location  within  the  areas  marked  for  the  landing  gear.  All  the  movement  of  aircraft  will  be 
conducted  in  a  highly  precautions  and  coordinated  manner.  Team  members  will  have  to  follow  standard 
operating  procedures. 

Team  project  title:  Aircraft  operation 
Number  of  members  in  a  team:  3 

Description:  Given  manufacturers'  operating  instructions,  team  will  locate,  select,  connect,  and  operate 
ground  support  equipment.  Team  will  start  and  operate  engine  through  normal  operating  range  and  perform 
I  shut  down  procedures. 


Course:  Assembly  and  Rigging 

Team  project  title:  Installing  flight  control 
Number  of  members  in  a  team:  4 

Description:  Team  members  will  identify  appropriate  service  manuals,  tools,  equipment,  and  forms.  Team 
members  will  assign  roles  to  remove,  inspect,  repair,  and  reinstall  one  flight  control  and  make  required 
maintenance  record  entries.  All  work  performed  needs  to  meet  manufacturers'  specifications.  Team  members 
will  play  the  role  of  inspector,  buy-back  inspector,  and  maintenance  personnel. 

Team  project:  Installing  vertical  stabilizer 
Number  of  members  in  team:  4 

Description:  Team  members  will  identify  appropriate  service  manuals,  tools,  equipment,  and  forms.  Team 
members  will  assign  roles  to  remove,  inspect,  repair,  and  reinstall  vertical  stabilizer  and  make  required 
maintenance  record  entries.  All  work  performed  needs  to  meet  manufacturers'  specifications.  Team  members 
will  play  the  role  of  inspector,  buy-back  inspector,  and  maintenance  personnel. 

Team  project:  Aircraft  control  rigging  (different  sub-systems) 

Number  of  members  in  team:  3 

Description:  Given  an  aircraft  with  cable  operated  flight  control  system,  service  manuals,  tools,  and 
equipment.  The  team  will  have  to  coordinate  work  and  assign  roles  to  inspect  the  system  for  proper  rigging, 
record  the  discrepancy,  and  make  repairs,  rig  the  flight  controls,  and  record  the  work  The  members  will  play 
the  role  of  a  inspection  and  maintenance  crew  on  a  rigging  check. 
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Year  2 

Course:  Utility  and  Warning  Systems 


Team  project  title:  Position  Indicating  and  Warning  Systems 
Number  of  members  in  a  team:  4 

Description:  Given  an  aircraft  with  retractable  landing  gear  and  position  indicating  and  warning  systems, 
ground  support  equipment,  and  the  manufacturers'  maintenance  and  service  instructions,  the  team  will  have 
members  with  assigned  roles  of  an  inspector,  buy-back  inspector,  and  maintenance  personnel.  The  team  will 
first  perform  an  operational  check  of  the  landing  gear,  inspect  components  of  the  position  indicating  and 
warning  system  (inspectors),  troubleshoot  and  repair  malfunctions  (maintenance  crew),  and  ensure  that  the 
work  meets  standards  (buy-back  inspector). 

Course:  Landing  Gear  Systems 

Team  project  title:  Aircraft  Jacking 
Number  of  members  in  a  team:  4 

Description:  Given  an  aircraft  with  operational  retractable  landing  gear,  manufacturers'  service  manuals; 
other  information,  and  ground  support  equipment,  the  team  will  have  to  assign  roles  and  coordinate  work  to 
accomplish  the  following:  jack  the  aircraft,  check,  inspect,  repair,  and  service  the  landing  gear  so  that  work  is 
accomplished  within  the  allowed  time  frame.  The  team  will  have  to  ensure  that  the  operation  of  the  systems 
and  the  manufacturers'  adjustment  procedures  are  followed  precisely  and  that  the  system  meets  "return-to- 
service"  standards. 

Course:  Airframe  Inspection 

Team  Project:  Airframe  Inspection  and  Maintenance 
Number  of  members  in  a  team:  4 

Description:  Given  an  operational  aircraft  ground  support  equipment  and  manufacturers'  service  manuals, 
the  team  will  have  members  with  assigned  roles  of  an  inspector,  buy-back  inspector,  and  maintenance 
personnel.  The  inspector  (first  team  member)  will  perform  an  annual  inspection  of  the  aircraft,  record 
conditions  at  the  time  of  inspection,  and  make  the  appropriate  aircraft  record  entries  to  communicate 
information  to  other  members  of  the  team  (maintenance  crew  consisting  of  2  team  members).  Team 
members  responsible  for  maintenance  activities  will  conduct  maintenance  and  have  it  inspected  by  another 
inspector  (fourth  member  of  the  team)  to  ensure  that  the  maintenance  work  meets  standards. 

Course:  Turbine  Engine  Overhaul 

Team  Project:  Engine  Overhaul 
Number  of  members:  4 

Description:  Given  a  turbojet  or  turboprop  engine,  manufacturers'  maintenance  manuals,  special  tools,  and 
shop  equipment,  working  as  a  team,  the  team  will  disassemble  ,  clean,  inspect,  identify  repairs,  and 
reassemble  both  cold  and  hot  sections  of  the  engine  within  a  specified  time  frame.  All  activities  and  practices 
will  be  performed  in  accordance  with  manufacturers'  maintenance  instructions. 

Team  Project:  Engine  Removal  and  Installation 
Number  of  members:  4 
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Description:  Given  an  aircraft  with  an  operational  turbojet  engine,  manufacturers'  maintenance  manuals,  and 
engine  removal  and  installation  equipment,  working  as  a  team,  the  team  will  perform  the  engine  removal  and 
reinstallation  procedures  to  meet  manufacturers'  standards  and  within  the  allocated  time  frame. 


Course:  Reciprocating  Engine  Overhaul 

Team  Project:  Engine  Overhaul 
Number  of  members:  4 

Description:  Given  a  reciprocating  engine,  manufacturers'  maintenance  manuals,  and  special  tools  and  shop 
equipment,  working  as  a  team,  the  team  will  disassemble ,  clean,  inspect,  identify  repairs,  and  reassemble  the 
engine  within  a  specified  time  frame.  All  activities  and  practices  will  be  performed  in  accordance  with 
manufacturers'  maintenance  instructions. 


Team  Project:  Engine  Removal  and  Installation 
Number  of  members:  4 

Description:  Given  an  aircraft  with  an  operational  reciprocating  engine,  manufacturers'  maintenance  manuals, 
and  engine  removal  and  installation  equipment,  working  as  a  team,  the  team  will  perform  the  engine  removal 
and  reinstallation  procedures  to  meet  manufacturers'  standards  in  the  allocated  time  frame. 
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The  Federal  Aviation  Administration  (FAA)  is 
committed  to  exploring  ways  of  restructuring  the 
regulatory  process  as  it  pertains  to  training, 
qualification  and  certification  of  advanced  skills 
(specialties)  in  the  aviation  maintenance  industry. 
They  recognize  a  need  for  a  flexible,  forward  looking 
and  more  efficient  system,  geared  to  the  rapid 
technological  and  industry  changes  taking  place  as 
we  approach  the  21st  century.  This  project  will 
evaluate  the  issues,  analyze  pertinent  information  and 
present  a  plan  for  development  of  such  a  system. 
Included  will  be  an  evaluation  of  both  US  and 
international  aviation  maintenance  technical  training 
and  qualification  standards,  and  certification  systems 
in  other  industries  that  require  skill  level  standards. 

Pertinent  information  from  other  studies,  such  as 
Pilot  and  Aviation  Maintenance  Technician  Blue 
Ribbon  Panel,  Aviation  Maintenance  Technician  Job 
Task  Analysis,  and  Human  Resources  in  the 
Canadian  Aircraft  Maintenance  Industry,  will  also  be 
included. 

The  system  will  be  based  on  evaluation  of  other 
industries  where  individuals  are  certified  to 
performance  standards  that  are  approved  and  kept 
current  by  recognized  industry  professional 
organizations.  Candidates  for  certification  are 
required  to  complete  specific  training  and 
competency  testing  approved  by  the  appropriate 
regulatory  agency  (the  FAA,  for  purposes  of  this 
project),  based  upon  the  specific  industry  group’s 
standards.  Examples  of  this  are  found  in  the 
certification  of  medical  technologists,  electronic 
technicians,  structural  welders,  and  various  other 
critical  safety  intensive  professions. 

The  initial  focus  of  this  study  is  to  research 
alternative  ways  to  develop  industry  input  for 
training  and  certification  standards  for  advanced 


aircraft  maintenance  skills.  This  project  will  provide 
a  basis  for  an  implementation  plan,  development  of 
the  standards  approval  process  and  the  selection  of 
technical  agencies  that  can  validate,  issue,  and 
maintain  these  standards.  The  possibility  of  forming  a 
national  aviation  industry  forum  that  would  provide 
information  for  industry  standards  development 
groups  and  advise  the  FAA  will  also  be  analyzed. 

This  effort  is  an  extension  of  the  regulatory  actions 
work  being  done  by  the  Federal  Aviation 
Administration  on  revising  rules  that  specify  the 
training  and  certification  of  aircraft  maintenance 
personnel  (Federal  Air  Regulations  Parts  147  and 
65).  Included  in  the  final  report  will  be  an  evaluation 
of  the  US  system  of  certification  for  Aviation 
Maintenance  Technicians  (AMT)  and  Aviation 
Repair  Specialists  (ARS). 

The  project  will  be  accomplished  in  two  phases.  This 
first  phase,  that  began  in  July  of  1994  and  completes 
at  the  end  of  March  1995,  will  include  investigation, 
information  and  data  gathering.  The  second  phase, 
April  through  December  of  1995,  will  focus  on 
development  of  proposals  and  the  final  report. 


itiiliiliiiiliiEiSlil 


There  is  increasing  evidence  that  validates  FAA 
and  Industry  concern  that  the  current  background 
information  and  industry  input  into  the  FAA  system 
for  training,  qualification  and  certification  of  aviation 
maintenance  personnel  may  be  insufficient.  There  is 
also  concern  that  the  FAA  certification  process  is  not 
geared  for  rapid  revision  and  technical  updates. 
Regulator  actions  have  not  kept  pace  with  changing 
aviation  technology  and  the  industry’s  maintenance 
skill  requirements.  These  concerns  are  focused  on 
persons  certified  as  Airframe  and  Powerplant 
Mechanics  (A&P),  and  Repairmen,  as  prescribed  in 
FAR  Part  65.  There  must  be  sufficient  input  to  ensure 
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that  aviation  maintenance  personnel  will  continue  to 
meet  the  current  and  future  needs  of  continuing  air 
worthiness.  It  is  necessary  to  explore  means  that  will 
enhance  the  role  of  the  industry’s  technical 
leadership  working  together  with  the  FAA  to  keep 
the  system  current. 

In  November  1989,  a  joint  industry  /  FAA  part  65 
review  group  was  formed  to  evaluate  and  review 
certification  requirements  for  mechanics  and 
repairmen.  The  review  group’s  objective  was  to 
develop  and  present  a  unified  position  on 
recommended  changes  to  part  65.  The  group  was 
composed  of  representatives  from  several  aviation 
associations  and  was  coordinated  by  the  Professional 
Aviation  Maintenance  Association  (PAMA).  FAA 
interests  were  represented  by  the  Aircraft 
Maintenance  Division  (AFS-300)  of  the  FAA. 

After  conducting  a  series  of  panel  discussions 
throughout  the  United  States,  the  Industry  /  FAA 
Part  65  Review  Group  Working  Paper  was  published 
in  January  of  1991.  This  paper  presented  the  issues 
on  which  there  was  general  agreement  and  those 
issues  that  the  group  believed  would  require  further 
discussion. 

During  1991,  the  FAA  also  conducted  both  a 
historical  review  of  part  65,  subparts  D  and  E,  and  a 
survey  of  FAA  regional  offices  on  the  certification  of 
mechanics,  holders  of  inspection  authorizations,  and 
repairmen.  Results  of  both  the  historical  review  and 
the  regional  office  survey  showed  clear  support  for  a 
full  review  and  update  of  part  65. 


Another  major  reason  for  review  and  revision  of  the 
Aircraft  Mechanic  and  Repairman  regulation  is  based 
upon  the  level  of  professionalism  in  these  career 
fields.  The  Pilot  and  Aviation  Maintenance 
Technician  Blue  Ribbon  Panel  Report  pointed  out 
that  the  US.  Department  of  Labor  Dictionarv_of 
Occupational  Titles  lists  aircraft  mechanics  and 
repairers  as  semi-skilled.  The  panel  recommended 
that  this  be  reviewed.  The  FAA  believes  it  is 
necessary  to  increase  the  level  of  professionalism 
within  these  occupations  and  have  Aviation 
Maintenance  Technicians  and  Aviation  Repair 
Specialists  recognized  as  highly  skilled. 


The  Pilot  &  Aviation  Maintenance  Technician 
Blue  Ribbon  Panel  Report  (Aug.  1993)  explains: 
The  majority  of  new- hire  AMTs  come  from  FAA- 
certificated  AMT  schools,  where  they  have  15  to  18 
months  of  structured  training  in  a  variety  of  subjects. 
Although  the  FAA  recently  revised  the  curriculum 
requirements  for  these  schools,  the  new  curriculum 
remains  broad-based  to  fit  a  variety  of  technical 
disciplines,  and  it  may  not  give  AMTs  the  skills  and 
competencies  needed  to  maintain  the  increasingly 
sophisticated  transport  category’  aircraft.  Therefore, 
new-hire  AMTs  working  on  newer  aircraft  will  have 
to  master  skills  that  many  AMT  schools  do  not  offer, 
if  they  are  to  become  productive  members  of  air 
transportation  teams. 

Thus,  the  industry’  will  face  a  problem  with  AMTs 
similar  to  the  problem  with  pilots:  A  decreasing 
supply  of  qualified  AMTs,  combined  with  increasing 
skill  and  experience  requirements,  will  yield  a  deficit 
not  in  the  number  ofrhinimally  qualified  individuals 
but  in  the  number  with  the  necessary  skills  and 
experience.  This  gap  will  have  to  be  bridged  by 
additional  focused  and  specialized  training.  Europe 
and  Asia  are  effectively  addressing  the  future  skills 
shortages  and  becoming  stronger  competitors, 
causing  dramatic  increases  in  the  amount  of  U.  S. 
work  done  in  foreign  repair  stations. 

The  aviation  industry  will  continue  moderate  growth 
well  into  the  next  century.  At  the  same  time  the 
forces  of  competition  in  the  de-regulated  air 
transportation  environment  mandate  lowering  prices 
to  the  consumer,  with  a  resulting  focus  on  lowering 
operating  costs  and  the  need  to  optimize  maintenance 
processes  and  practices.  This  competition  has  also 
spun'ed  the  development  of  improved  aircraft 
technology  and  operational  efficiency.  Today’s 
aircraft  are  significantly  more  sophisticated,  from 
both  a  materials  and  systems  standpoint,  than  those 
built  and  certified  when  the  current  maintenance 
regulations  were  developed. 

The  industry  finds  itself  in  a  challenging  situation. 
Significant  changes  are  being  made  by  air  carriers 
with  respect  to  internal  maintenance  programs  and 
the  contracting  of  second  and  third  party  agencies  to 
maintain  and  modify  their  fleets.  In  the  past  most 
carriers  completed  a  majority  of  maintenance  work 
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in-house,  but  it  is  now  often  more  efficient  and  cost 
effective  for  them  to  have  major  work  and 
modifications  accomplished  by  others.  The  numbers 
of  aircraft  that  are  owned  by  leasing  companies, 
maintained  by  various  agencies  worldwide  and 
moved  from  operator  to  operator,  have  dramatically 
increased.  Along  with  the  international  aspects  of 
movement  of  aircraft  within  different  fleets  and 
maintenance  programs,  is  the  dramatic  increase  in  the 
number  of  foreign  certified  repair  stations  and 
maintenance  work  begin  done  “off  shore.”  These 
factors,  mixed  with  numerous  technology  changes, 
have  increased  the  complexity  of  aircraft 
maintenance.  All  of  this  has  created  both  FAA  and 
industry  concern. 

The  present  maintenance  regulatory  system  is 
cumbersome;  it  was  not  designed  for  rapid  change. 
Changes  due  to  new  technology  and  the  dynamics  of 
the  global  business  environment  make  it  difficult  for 
the  rules  that  regulate  training  and  qualification  to 
keep  pace.  Finding  methods  that  will  allow  for  a 
more  responsive  regulatory  system  under  the  rules, 
while  at  the  same  time  focusing  on  international 
harmonization,  is  essential. 


The  ARAC  was  established  (56  CFR  2190, 
January  22,  1991)  to  assist  the  FAA  in  the 
rulemaking  process  by  providing  input  from  outside 
the  Federal  Government  on  major  regulatory  issues 
affecting  aviation  safety.  This  process  is  designed  to 
provide  opportunity  for  those  groups  in  the  industry 
who  are  significantly  affected  by  rulemaking  to 
become  involved  in  the  process.  Since  affected 
parties  are  involved  in  the  process  the  rules  produced 
should  be  more  complete,  require  less  direct  effort  on 
the  part  of  the  FAA,  have  few  elements  of  contention 
from  the  public  when  published  and  move  rapidly 
from  initial  review  to  final  effectivity. 

The  ARAC  includes  representatives  of  air  carriers, 
manufacturers,  general  aviation,  organized  labor 
groups,  universities,  associations,  airline  passenger 
groups,  and  the  general  public.  Formation  of  the 
ARAC  has  given  the  FAA  additional  opportunities  to 
solicit  information  directly  from  all  elements  of  the 
industry.  There  are  several  working  groups  under 
ARAC  that  meet  to  exchange  ideas  about  proposed 


rules  and  existing  rules  that  should  be  either  revised 
or  eliminated. 

Formed  initially  in  November  1989,  as  the  Joint 
Industry  /  FAA  Part  65  Review  Group,  the  Aviation 
Rule  Making  Advisory  Committee  Working  Group 
for  FAR  Part  65  (ARAC  -  65)  has  been  meeting 
officially  since  May  24,  1991.  The  working  group  is 
made  up  of  representatives  from  aviation  industry 
professional  organizations,  aviation  training 
providers,  air  transport  labor  unions,  industry 
representatives,  and  the  general  public.  One  of  the 
major  objectives  of  ARAC  is  to  shorten  the  time  it 
takes  to  revise  regulations  by  involving  all  interested 
parties  in  the  process.  This  working  group  is 
responsible  for  regulatory  review  and  recommending 
changes  to  FAR  Part  65,  Certification  of  Airmen 
Other  Than  Flight  Crew  Members,  specifically  the 
portion  regulating  mechanics,  mechanics  holding 
inspection  authorizations  and  repairmen.  Their 
efforts  have  yielded  significant  changes  and  upgrades 
to  FAR  Part  65  which  are  scheduled  to  be  released  as 
a  Notice  of  Proposed  Rule  Making  (NPRM)  in  the 
winter  of  1995  /  1996.  Substantive  recommended 
changes  to  Part  65  are  outlined  in  Appendix  12- A. 

If  the  process  remains  on  schedule,  the  new  rule 
(consolidated  as  FAR  Part  66)  may  become  effective 
in  mid- 1998.  This  means  that  the  process  to  review 
FAR  part  65  will  have  been  in  the  works  for  nine 
years.  The  process  of  evaluating  and  recommending 
changes  to  Federal  Air  Regulations  remains  long  and 
cumbersome. 


The  ARAC  -  65  working  group  has  discussed  and 
evaluated  a  significant  number  of  issues 
regarding  advanced  certification.  The  group’s 
consensus  is  that  a  new  process  needs  to  be 
developed  and  that  the  research  project  described 
herein  is  a  necessary  step  toward  reaching  that 
objective.  Suggested  changes  in  the  current  Airframe 
and  Powerplant,  and  Repairman  Certificates  reflect 
the  complexity  of  today’s  technology,  and  represent  a 
wide  range  of  input  toward  the  development  of  an 
advanced  certification  process.  Since  the  members 
represent  a  large  cross  section  of  the  industry,  their 
views  may  be  considered  as  a  reasonable 
representation  of  the  industry’s  thinking  on  this  issue. 
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The  Airframe  and  Powerplant  Certificate  (A&P)  is 
based  on  a  broadly  focused  1900  hour  minimum 
curriculum  specified  in  FAR  Part  147.  The  Airframe 
or  Powerplant  privileges  of  the  certificated  may  be 
issued  separately  under  the  current  rule.  The 
certification  under  the  new  Part  66  rule  will  be  titled 
Aviation  Maintenance  Technician  (AMT),  will 
include  a  common  set  of  privileges  and  be  issued 
only  as  a  singe  certificate.  Advanced  certification 
will  be  provided  with  the  addition  of  the  Aviation 
Maintenance  Technician  -  Transport  (AMT-T) 
privilege.  This  certification  will  require  an  additional 
curriculum,  approximately  600  hours  above  the  1900 
hours  required  in  the  current  rule,  that  is  specific  to 
the  current  technology  of  Part  25  (air  transport  fixed 
wing)  and  29  (air  transport  rotor  craft)  certified 
aircraft,  along  with  additional  competency  testing. 
Persons  may  select  the  level  of  certification  for  which 
they  wish  to  qualify.  An  AMT-T,  however,  will  be 
required  to  return  transport  category  aircraft  to 
service  once  the  rule  becomes  final. 

Through  the  creation  of  the  AMT-T  operators  of 
aircraft  certificated  under  FAR  Parts  25  and  29 
(commercial  airplanes  and  helicopters)  will  be 
assured  that  the  holder  of  an  AMT-T  certificate 
possesses  the  knowledge  and  skill  to  approve  these 
aircraft  for  return  to  service  (or  “sign  off’  of  a 
maintenance  release).  This  will  allow  operators  to 
employ  aviation  maintenance  personnel  who  will 
more  quickly  meet  the  requirements  of  their 
operating  environment  without  having  to  attend 
extensive  operator-sponsored  training  programs 
before  performing  maintenance  on  transport  aircraft. 
Operators  would  be  able  to  focus  their  training  on 
aircraft  type,  aircraft  differences,  modifications,  and 
technology  upgrade  of  transport  aircraft.  Aviation 
maintenance  technician  training  schools  (certified 
under  FAR  Part  147)  would  be  able  to  focus  on  the 
fundamental  concepts  and  basic  skills  of  aviation 
maintenance.  They  would  also  have  the  option  of 
providing  the  additional  knowledge  and  skill  required 
for  AMT-T  certification. 

The  Repairman  Certificate  is  cun'ently  issued  to  an 
individual  for  a  specific  maintenance  task(s), 
appliance  or  component  repair  /  overhaul,  for  FAR 
Part  121  or  135  Operators  under  subpart  J  and  L, 
Fixed  Base  Operators  (FBO),  or  FAR  part  145 
Certified  Repair  Stations.  They  are  also  issued  by  the 
FAA  to  those  individuals  constructing  amateur-built 
aircraft  for  their  own  non-pommercial  use.  The 
Repairman  Certificate  process  has  been  significantly 
revised  under  the  new  FAR  Part  66. 


The  new  certification  will  grant  specific  repair  and 
maintenance  privileges  to  Aviation  Repair  Specialists 
(ARS).  The  ARS  will  be  issued  in  three  categories, 
defined  as  follows: 

1.  ARS-I  -  May  be  issued  by  the  FAA  upon 
completion  of  an  industry  developed  standards- 
based  training  curriculum  and  appropriate 
competency  testing  and  /  or  validation  to  an 
individual.  The  individual  who  has  earned  such 
certification  may  only  exercise  these  privileges 
while  employed  at  a  Certified  Repair  Station, 
Part  121  or  135  Operator.  This  provides  limited 
portability  for  this  level  of  certification.  The  skill 
areas  where  ARS-I  certification  will  be  granted 
are  to  be  determined  based  on  the  outcome  of 
this  project,  and  the  Job  Task  Analysis  project 
being  completed  by  Northwestern  University’s 
Transportation  Research  Center.  Also  included 
may  be  areas  with  current  standards  such  as  non¬ 
destructive  inspection  (NDI). 

2.  ARS-II  -  Issued  as  a  replacement  for  today’s 
Repairman  Certificate  and  will  be  issued  under 
similar  regulations. 


3.  ARS-III  -  Issued  by  the  FAA  to  amateur 
builders,  producing  “home  built”  aircraft  for 
their  own  non-commercial  use,  as  in  the  past. 


The  working  sessions  of  ARAC-65  generated 
presentations  from  various  industry  groups  that 
stimulated  discussion  regarding  advanced 
certification  and  appropriate  skill  areas.  No  firm 
decisions  were  made  specifying  what  functional  areas 
may  be  finally  selected  for  advanced  certification.  It 
was  concluded  that  there  may  eventually  be  new 
ARS-I  categories  beyond  those  listed  below.  The 
group  agrees  with  the  FAA  that  training, 
qualification,  and  certification  will  be  based  on 
nationally  and  internationally  recognized  standards 
developed  by  the  aviation  maintenance  industry.  The 
following  skill  areas  have  been  selected  as  those  that 
will  be  considered  for  advanced  certification 
standards  and  ARS-I  certification: 

®  Aircraft  Electronics  (Avionics) 

°  Composite  Structural  Repair 
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•  Non-destructive  Inspection 

•  Metal  Structures  Repair 

•  Balloon  and  Glider  Repair 

As  rule  making  evaluation  and  change  continues, 
there  may  be  other  skill  areas  identified  and  added  to 
the  list.  There  has  been  a  good  deal  of  work 
completed  toward  development  of  training, 
qualification  and  certification  standards  in  the 
following  areas: 

12.5.1  Aircraft  Electronics 

In  its  broadest  definition,  aviation  electronics,  also 
known  as  avionics,  encompasses  all  aircraft  electrical 
/  electronic  systems  and  their  components.  The  term 
“avionics”  now  goes  beyond  a  more  basic  definition 
that  once  included  only  communication,  navigation 
and  auto-flight  systems. 

One  of  the  major  changes  in  today’s  aircraft  is  the 
extensive  use  of  digital  electronic  data  processors, 
computers,  electronic  controls,  and  fly-by-wire 
technology.  Aircraft  have  become  fully  integrated 
from  a  systems  standpoint.  While  additional 
emphasis  has  been  placed  on  avionics  in  the 
proposed  Aviation  Maintenance  Technical  - 
Transport  (AMT-T)  rating  in  FAR  Part  66,  there  is  a 
large  group  within  the  aviation  industry  that  strongly 
supports  an  ARS  - 1  level  avionics  technician 
certification.  Maintenance  and  alteration  of  these 
systems  requires  a  highly  specialized  set  of  skills  and 
knowledge  that  go  beyond  AMT  and  AMT-T 
requirements. 

The  Association  for  Avionics  Education  (AAE),  with 
the  support  of  the  Aircraft  Electronics  Association 
(AEA),  is  in  the  process  of  developing  a  training  and 
qualification  standard  for  Aviation  Electronics 
Technicians.  Their  working  documents  have  been 
presented  to  ARAC-65  on  two  occasions  for  review 
and  comment.  The  ARAC-65  group  has  concluded 
that  there  will  not  be  a  separate  avionics  rating  as  part 
of  AMT  or  AMT-T  certification.  They  have 
encouraged  AAE  to  continue  with  their  standards 
development  process,  addressing  aircraft  electronics 
as  an  ARS-I  certification. 


12.5.2  Composite  Structural  Repair 

Composites  are  non  metallic  structures  that  include 

materials  such  as  fiberglass,  carbon  fiber,  kevlar  , 
and  graphite  filament.  They  are  usually  chemically 
compounded  or  laminated  with  resins  and  bonded  to 
metal,  or  other  composite,  support  structures  with 
adhesives  to  make  light-weight,  non-corroding,  high- 
strength  aircraft  structural  components.  They  are 
often  formed  and  cured  under  heat  and  vacuum. 
Special  equipment  and  working  environments  are 
often  required  to  construct  or  repair  composite 
structures.  Special  skills  are  required  as  improper 
handling  or  repair  techniques  can  cause  extensive 
damage  and  the  materials  themselves  can  create  both 
worker  health  and  environmental  hazards. 

Most  indicators  point  toward  the  increased  use  of 
composite  materials  in  aircraft  construction, 
particularly  transport  aircraft.  Some  aircraft  currently 
in  production  are  “all  composite.”  It  has  become  a 
very  complex  and  highly  specialized  segment  of 
aviation  maintenance.  The  knowledge  and  skills 
necessary  for  composite  maintenance  require  an 
expertise  beyond  the  AMT  and  AMT-T  certification 
requirements. 

The  Commercial  Aircraft  Composite  Repair 
Committee  (CACRC),  sponsored  by  the  Society  of 
Automotive  Engineering  (SAE),  is  in  the  process  of 
formulating  a  standard  for  this  skill  area.  The  format 
from  Air  Transport  Association  (ATA)  Specification 
105  (Non  Destructive  Inspection)  is  being  used  as  a 
model.  The  CACRC  group  has  gained  international 
stature,  based  in  representation  from  the  European 
aviation  maintenance  community.  They  have  been 
meeting  for  over  two  years  developing  their 
standards  and  have  made  a  good  deal  of  progress 
with  the  document.  The  group  is  close  to  the  release 
of  a  draft  that  will  include  guidelines  for  composites 
materials  handling,  preventative  maintenance, 
inspection,  repair,  alteration  /  fabrication,  and 
protective  coatings. 

12.5.3  Non-destructive  Inspection 

Non-destructive  inspection  (NDI)  has  become  a  very 
highly  specialized  skill  area  that  requires  the  use  of 
sophisticated  tooling  and  diagnostic  equipment  for 
the  evaluation  of  defects  and  flaws.  Technology 
ranges  from  magnetic  particle  and  dye  penetrant 
methods  through  x-ray,  ultrasonic,  eddy  current  and 
some  currently  emerging  technologies.  The 
technician  is  responsible  for  the  setup  and  operation 
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of  these  systems,  plus  the  reading  and  interpretation 
of  their  output.  Competency  in  non-destructive 
testing  requires  a  high  degree  of  both  knowledge  and 
skill.  Proficiency  also  requires  a  good  deal  of  hands- 
on  practice  and  recurrent  training. 

There  have  been  recent  improvements  in  non¬ 
destructive  inspection  technology.  Sandia  Laboratory 
in  Albuquerque,  New  Mexico  has  a  dedicated  facility 
and  a  staff,  complete  with  air  transport  category 
aircraft,  for  the  development  and  application  of  non¬ 
destructive  testing  technology.  There  are  also  human 
factors  studies  underway  that  are  focused  on 
improving  visual  inspection  tools  and  processes. 
These  studies  are  expected  to  produce  human 
engineering  results  that  will  enhance  techniques, 
therefore  benefiting  the  technician’s  ability  to 
conduct  visual  inspections. 

The  Air  Transport  Association  Non  Destructive 
Inspection  Sub  Committee  has  developed 
Specification  105,  Guidelines  for  Non  Destructive 
Inspection.  The  document  includes  training  curricula 
for  the  various  NDI  processes  and  associated 
inspection  techniques.  Also  included  are  qualification 
standards  for  NDI  personnel.  ATA  Specification  105 
represents  a  quality  body  of  work  that  was  developed 
with  input  from  all  elements  of  the  aviation 
manufacturing  and  maintenance  industry. 

The  American  Society  for  Non  Destructive  Testing 
standards  have  been  in  place  for  a  number  of  years. 
They  are  kept  current  with  state  of  the  art  processes 
and  emerging  technology.  These  standards  specify 
training,  qualification  and  certification  of  NDI 
specialists  in  each  of  the  NDI  processes,  from  the 
basics  through  the  most  complex  radiography.  Their 
standards  are  recognized  by  several  industries  other 
than  air  transport  and  they  are  considered  as  the 
model. 

While  there  are  two  other  standards  that  are 
recognized  in  the  non  destructive  inspection 
discipline,  the  aviation  industry  recognizes  ATA 
Specification  105  and  ASNT  as  the  baseline.  One,  or 
both,  of  these  could  become  standards  that  are 
accepted  by  the  FAA  for  ARS-I  certification. 

12.5.4  Metal  Structures  Repair 

Aircraft  structure  maintenance,  modification  and 
repair  is  an  area  of  increasing  focus  and  concern. 
Several  factors  are  causing  changes  in  the  nature  of 
work  content  and  specialization  of  personnel  within 


this  element  of  the  maintenance  industry. 

Specifically,  the  need  to  reduce  operating  costs  is 
motivating  the  air  transport  community  to  conduct 
business  differently: 

•  Increasing  amounts  of  modification  and  repair 
work  (up  to  and  including  D  check  level)  is 
being  accomplished  by  second  and  third  party 
maintenance  providers. 

•  The  number  of  aircraft  classified  as  aging  is 
increasing.  By  definition  and  structural  status, 
these  aircraft  require  extensive  structural 
inspections,  repairs  and  modifications  in  order  to 
remain  airworthy. 

•  The  size  of  the  leased  aircraft  fleet  is  at  an  all 
time  high,  with  continued  growth  forecast  for  the 
future.  These  aircraft  move  from  operator  to 
operator  and  are  maintained  by  various  AMOs 
around  the  world. 

®  Many  airframe  specialists  are  not  certified 

because  they  are  not  required  to  return  aircraft  to 
service.  They  specialize  in  structures  repair,  and 
are  not  Airframe  and  /  or  Powerplant  certificate 
holders.  They  usually  work  at  AMOs  and  are 
covered  under  FAR  Part  145  repair  station 
certification. 

•  Since  a  large  percentage  of  the  work  done  by 
second  and  third  party  maintenance  providers  is 
competitively  bid,  workload  for  these  operations 
is  cyclical  with  variable  staffing  demands.  This 
has  created  a  significant  number  of  temporary 
contract  aircraft  maintenance  personnel  agencies. 
The  workers  in  this  field  are  assigned  by  contract 
to  operations  worldwide  that  need  maintenance 
staff  They  are  transient,  moving  from  company 
to  company  and  place  to  place  as  needed.  Most 
of  these  workers  are  non  certificated  structures 
mechanics  with  training,  qualifications  and 
backgrounds  that  are  supported  only  by  resumes 
and  word  of  mouth. 

An  independent  Structures  Repair  Committee  (SRC) 
was  formed  by  several  participants  involved  in  the 
CACRC  is  also  in  the  process  of  developing  a 
standard  for  aircraft  metal  structures  repair 
specialists.  The  intended  purpose  is  to  create  a 
document  that  will  describe  the  training, 
qualifications,  and  certification  of  aircraft  metal 
structures  repair  specialists  as  an  ARS-I.  They  are  at 
about  the  same  point  of  development  with  the 
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structures  repair  standards  as  CACRC  is  with  the 
composite  materials  repair  standard.  Meetings  to 
continue  development  work  have  been  held  as 
recently  as  February  14,  15,  and  16,  1995.  Progress 
continues  to  be  made  and  this  effort  will  continue  to 
be  evaluated  as  a  part  of  this  project. 

There  is  a  strong  body  of  thought  within  the  industry 
that  aircraft  structures  repair  should  be  covered  by  a 
standard  and  require  certification  at  the  level  required 
to  meet  ARS-I  certification.  This  was  demonstrated 
in  results  from  a  recent  survey  that  is  discussed  in 
section  12.6. 

12.5.5  Balloon  and  Glider  Repair 

Balloon  maintenance  and  repair  although  a  relatively 
small  segment  of  the  industry,  is  currently  asking  for 
specialty  status  and  fits  under  the  ARS-I  concept. 
Balloons  are  not  true  airframes,  nor  do  they  have 
conventional  powerplants,  yet  under  current 
definitions  they  fall  under  the  same  FAA  rules  as 
standard  aircraft.  Balloons  must  be  maintained  by  A 
&  P  mechanics  and  lA’s  under  FAR  Part  91  as 
general  aviation  aircraft.  They  may  also  be  repaired 
by  repairmen  in  certified  repair  stations.  It  is  the 
contention  of  many  in  balloon  operations  and 
maintenance  that  safety  is  compromised  from  lack  of 
specific  training,  qualification  and  certification 
standards.  Commercial  operators  contend  that  there 
should  be  a  set  of  minimum  standards  for  both  repair 
facilities  and  maintenance  personnel. 

A  proposed  standard,  supported  by  several  operators, 
was  presented  to  the  FAA  at  one  of  the  Maintenance 
Regulatory  Reviews  in  December  of  1989.  It 
included  a  minimum  equipment  list  for  hot  -  air 
balloon  repair  stations,  and  a  minimum  task  list 
(qualifications)  for  certified  balloon  repairmen. 

The  FAA  concurs  with  the  direction  taken  by  the 
balloon  industry  and  will  encourage  the  completion 
of  standards  that  may  be  accepted  for  ARS-I 
certification. 

While  there  is  no  specific  information  available  at 
this  time,  the  FAA  has  also  recognized  that  a  similar 
situation  to  the  balloon  sector  also  exists  in  the  glider 
maintenance  and  repair  sector.  Means  of  having  this 
sector  develop  acceptable  ARS-I  certification 
standards  will  be  explored. 


12.5.6  Other  Potential  Skill  Areas 

There  is  general  agreement  in  ARAC  and  the  FAA 
that  the  skills  listed  above  represent  the  areas  of 
primary  need  and  focus.  Continuing  research  and 
investigation  during  the  second  phase  of  this  project 
will  focus  on  these  and  other  skill  areas  that  are 
potential  candidates  for  specialist  certification. 
Working  with  Northwestern  University’s 
Maintenance  Job  Task  Analysis  team’s  initial  data 
should  also  serve  to  verify  what  the  ARAC  has 
accomplished.  This  data  should  also  illuminate  any 
other  obvious  areas  where  specialist  certification 
needs  to  be  considered. 


A  member  of  CACRC,  with  agreement  from  the 
group,  conducted  a  survey  of  a  cross  section  of 
FAR  145  repair  station  operators.  This  survey  was 
random,  not  intended  to  be  formal  nor  statistically 
validated.  However,  it  does  provide  worthwhile 
information,  available  nowhere  else,  on  the  subject  of 
advanced  certification  for  specific  skill  areas  in 
certified  repair  stations. 

The  survey  was  sent  to  40  Part  145  repair  stations, 
selected  from  the  World  Aviation  Directory  (WAD), 
who  perform  work  on  large  transport  category 
aircraft.  It  asked  for  information  concerning  the  array 
of  technicians  employed  at  these  facilities.  The 
questions  targeted  A  &  P  certified  mechanics,  and  the 
four  potential  specialist  groups  considered  for  ARS-I 
certification  by  the  ARAC-65  working  group. 
Twenty-three  of  the  repair  stations  responded,  which 
at  over  57%  is  a  very  good  response.  They  were 
asked  to  provide  the  following  information: 

•  Total  number  of  technicians  employed 
•  Total  number  of  certified  A&Ps 
•  Total  non-certified  structural  /  sheet  metal 
technicians 

•  Total  number  of  Avionics  technicians 
•  Total  number  of  Avionics  technicians  with  FCC 
licenses 

•  Total  number  of  Avionics  technicians  holding 
repairman  certificates 
•  Total  number  of  NDI  technicians 
•  Total  number  of  Composites  technicians 
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e  What  type  of  maintenance  training  does  your 
company  offer? 

«  Would  the  company  be  better  served  by 
technicians  trained  to  industry  standards? 

®  Would  the  company  support  development  of 
specialist  ratings  in: 

-  Avionics 

-  Non  Destructive  Inspection 

-  Structures 

Unlike  the  major  air  carriers,  where  at  least  90%  of 
maintenance  personnel  hold  A&P  certificates,  the 
Part  145  operators  employ  maintenance  staff  where 
less  than  50%  hold  Ac&P  certification.  Structures 
repair  technicians  represented  almost  35%  of  the 
population  of  employees  covered  by  the  survey,  none 
with  certification  of  any  type.  It  was  also  interesting 
to  note  that  only  61%  of  the  respondents  conduct 
training  for  technicians  in  the  specialties  surveyed. 
This  points  out  that  there  could  be  a  significant  gap 
in  competencies  between  the  air  carrier  and  second  or 
third  party  maintenance  personnel. 

All  respondents  indicated  that  industry  standards  in 
the  specialties  listed  above  would  benefit  their 
operations.  The  survey  shows  that  there  is  interest 
within  the  industry  in  the  development  of  standards. 
Those  responding  were  fully  supportive  of  avionics 
and  NDI  standards  and  were  within  one  percentage 
point  of  full  support  for  composite  and  metal 
structures  repair. 

In  discussions  with  individuals  from  all  areas  of  the 
industry,  there  seems  to  be  general  agreement  that  the 
development  of  such  standards  is  a  worthwhile  and 
necessary  undertaking. 


Looking  at  systems  and  processes  by  which  other 
industries  and  disciplines  develop  and  maintain 
standards  for  training,  qualification  and  certification 
of  skills  will  provide  examples  of  how  this  may  best 
be  accomplished  in  the  aviation  maintenance 
industry.  A  broad  brush  snapshot  of  other  industries, 
with  both  technical  and  non-technical  knowledge  and 
skill  requirements,  has  shown  that  there  is  a  set  of 
consistent  characteristics.  There  are  two  general 
approaches  to  skill  and  knowledge  certification: 

®  Imposed  and  maintained  by  governmental 

agency  (Federal,  State,  County,  City  or  District) 
through  rules  and  regulations. 


®  Self-imposed  certification,  based  on  standards 
that  are  designed  to  maintain  specific  levels  of 
performance.  In  most  cases  the  development  of 
these  standards  and  the  resulting  training, 
qualification  and  certification  systems  are  under 
the  auspices  of  non-profit  professional 
organizations.  Such  standards  are  usually  put  in 
place  for  the  purpose  of  ensuring  public  safety, 
elevating  the  professional  standing  and  /  or 
perception  of  a  craft,  career  field,  or  profession, 
and  in  some  cases  to  avoid  or  preclude  imposed 
certification  /  regulation,  i.e.,  American  Welding 
Society,  Professional  Association  of  Diving 
Instruction,  etc. 

There  are  various  national  organizations  that  have 
developed  training  and  certification  standards,  for  a 
wide  range  of  skills,  that  are  in  continuous  use  today. 
Each  organization  has  a  board  of  directors, 
governors,  or  standards  committee,  consisting  of 
recognized  “senior”  experts  in  the  respective  fields. 
While  the  actual  skills  for  which  the  training  and 
certification  standards  have  been  developed  vary  a 
great  deal,  the  processes  by  which  they  were 
developed,  applied,  and  maintained  are  similar.  Some 
examples  of  these  organizations  and  information 
pertinent  to  their  successful,  currently  operational, 
training  qualification  and  certification  systems  are  as 
follows: 

American  Red  Cross  (ARCI 
While  far  removed  from  the  technical  world  of 
aviation  maintenance,  one  of  the  best  examples 
of  a  successful  training  and  certification 
process,  which  has  been  effective  for  nearly  a 
century,  is  the  method  used  by  the  American 
Red  Cross.  This  organization  has  a  solid 
training  and  certification  system  that  is 
recognized  around  the  world.  Their  national 
headquarters  establishes  and  maintains 
standards  for  training  and  certification  of 
various  public  safety  related  skills  such  as:  First 
Aid,  First  Aid  Instructor,  Jr.  Life  Saver,  Sr.  Life 
Saver,  and  Water  Safety  Instructor. 

The  organization  is  completely  self  contained 
and  accomplishes  all  training  and  certification 
through  a  comparatively  small  compensated 
staff  and  a  large  and  complex  national  network 
of  volunteers.  Many  organizations  recognize 
Red  Cross  certification  as  pre-requisite  for  other 
training,  such  as  Emergency  Medical 
Technician,  or  as  a  job  requirement  as  in  Life 
Guards  and  Swimming  Instructors. 
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American  Welding  Society  (AWS) 

The  FAA  does  not  require  additional 
certification  for  aircraft  construction  or  repair 
welding  beyond  the  Airframe  and  Powerplant 
ratings.  Based  on  the  most  recent  revision  of 
FAR  Part  147,  A&P  mechanics  must  be  able  to 
differentiate  between  acceptable  and 
unacceptable  welds,  but  are  no  longer  required 
to  demonstrate  welding  proficiency.  (The  state 
of  the  art  has  progressed  well  beyond  basic 
acetylene  gas  and  electric  arc  welding.)  Many 
airlines  and  repair  facilities,  however,  require 
welders  (especially  those  performing  “exotic” 
and  critical  welding)  in  component  and  engine 
repair  shops  to  be  AWS  certified. 

The  AWS  was  founded  in  1919  to  advance  the 
science,  technology  and  application  of  welding. 
It  is  a  non-profit  organization  that  conducts 
welder,  welding  inspector,  and  welding 
educator  certification  programs.  The  Society’s 
over  42,000  members  consist  of  educators, 
engineers,  researchers,  welders,  inspectors, 
technicians,  welding  foremen,  company 
officers,  and  supervisors.  Disciplines  include 
automatic,  semi-automatic  and  manual  welding, 
as  well  as  brazing,  soldering,  ceramics, 
robotics,  thermal  spraying  and  lasers.  (All  of 
these  processes  are  used  in  the  aviation 
maintenance  industry.)  Activities  include 
initiatives  in  research,  safety  and  health, 
education,  training,  business,  and  government 
liaison.  Their  standards  are  considered  as 
benchmarks  in  the  welding  craft.  They  also 
maintain  a  system  of  accredited  education  and 
test  facilities  in  the  fifty  States  and  overseas 
locations. 

An  example  of  their  system  and  the  process  that 
relates  to  advanced  certification  for  the  aviation 
maintenance  industry  is  their  Certified  Welder 
program  (similar  standards  exist  for  Welding 
Inspector  and  Welding  Educator  qualification 
and  certification).  The  Society’s  Certified 
Welder  Program  is  established  to  identify  all 
elements  necessary  to  implement  a  National 
Registry  of  Certified  Welders. 

The  four  key  elements  of  the  system  include: 

1.  Welder  performance  qualification 
standards. 

2.  Standard  welding  procedure  specifications. 


3.  Accredited  performance  qualification  test 
facilities. 

4.  AWS  welder  certification  requirements. 

The  purpose  of  the  Standard  for  AWS  Certified 
Welders  is: 

1 .  To  determine  the  ability  of  welders  to 
deposit  sound  welds  in  accordance  with 
standardized  requirements. 

2.  To  impose  sufficient  controls  on  the 
documentation  and  maintenance  of 
certification  to  allow  transfer  between 
employers  without  re-qualification,  where 
allowed  by  Standard  of  Contract 
documents. 

Specific  specialties  for  advanced  certification 
include:  Chemical  Plant,  Petroleum  Refinery 
Piping,  and  High  Rise  Construction. 

Application  for  certification  is  extensive  and 
includes  verification  of  background,  experience 
and  education.  They  also  require  medical 
certification  of  acceptable  visual  acuity 
completed  not  sooner  than  six  months  prior  to 
testing  and  certification. 

The  AWS  standards  are  well-defined  voluntary 
consensus  standards,  developed  in  accordance 
with  the  rules  of  the  American  National 
Standards  Institute  (ANSI),  They  provide  an 
excellent  basis  on  which  to  pattern  the 
development  of  standards  for  training, 
qualification  and  certification  of  aviation 
maintenance  skills. 

Radiological  Technologists  /  X-Ray 
Technicians 

The  system  of  training,  qualification  and 
certification  of  Radiological  Technologists  in 
the  state  of  California  is  typical  of  processes  for 
this  discipline  across  the  United  States. 

The  program  is  administered  by  California 
Health  Services,  Radiological  Health  Branch. 
This  organization  sets  the  standards  for  training 
and  curriculum  for  Radiological  Technologists. 
It  is  generally  a  2  or  3  year  program  conducted 
by  the  state’s  community  colleges.  Successful 
completion  of  such  a  program  qualifies  the 
learner  to  take  the  state  examination.  The 
examinations  are  conducted  by  Comprehensive 
Personnel  Services  (CPS),  a  for  profit 
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organization  that  conducts  these,  and  similar 
tests,  for  governmental  agencies.  CPS  only  does 
testing,  they  conduct  no  training  or  other  related 
activities. 

There  are  also  Limited  Permit  Technicians  who 
are  qualified  with  shorter  duration,  specific 
focus  courses,  often  taught  by  business  schools 
or  medical  technician  schools.  These  courses 
generally  certify  technicians  to  perform  X-rays 
on  specific  parts  of  the  body,  such  as  podiatry, 
chest,  etc.  They  are  qualified  through  on  the  job 
training,  and  certified  upon  successfully  passing 
a  state  administered  test. 

Board  Certified  Radiologists  (Physicians) 
automatically  receive  state  certification.  Other 
physicians  may  sit  for  and  pass  exams  to  gain 
certification. 

Schools  apply  to  the  California  Health  Services 
Administration  for  approval  of  their  programs 
by  completing  an  extensive  application  showing 
their  curriculum  content.  Oversight  is 
conducted  by  Inspectors  from  the  California 
Health  Services  staff.  Limited  Permit  Programs 
generally  receive  more  scrutiny  than  the 
programs  conducted  at  the  community  colleges. 

There  is  a  National  Society  of  Radiological 
Technologists  and  a  California  Registry  of 
Radiological  Technologists.  The  national 
organization  sets  the  pattern  for  standards  from 
which  the  California  program  is  adapted. 

Changes  are  a  regulatory  process  that  may  be 
driven  by  the  California  State  Legislature.  For 
instance,  there  is  current  interest  in  assuring 
quality  in  mammography.  This  is  also  being 
developed  as  a  new  advanced  certification 
category.  It  will  require  additional  training  and 
examination  after  initial  certification. 

Re-certification  is  required  every  2  years.  The 
re-certification  is  automatic  if  the  application  is 
timely.  A  continuing  education  requirement  will 
become  effective  in  July  of  1996. 

The  National  Society  of  Radiological 
Technologists  and  the  Society  of  Nuclear 
Medicine  conduct  conferences  that  often 
include  post  graduate  programs  (similar  to 
Inspector  Authorization  renewal  conducted  at 
PAMA  conferences).  These  groups  are  at  the 


level  of  industry  organizations  and  do  not 
develop  standards  for  training,  qualification  and 
certification. 

Emergency  Medical  Technicians  (EMT)  and 
Paramedics  -  (California) 

EMT  and  Paramedic  training  programs  are 
operated  under  standards,  generally  based  on 
national  guidelines,  but  developed  and 
maintained  by  individual  states.  It  is  also  a 
system  that  uses  partnership  between 
government  regulatory  agencies,  where  the 
public  and  private  educational  sector  provides 
the  training,  qualification  and  certification  for 
individuals  entering  a  specific  career  field. 

The  U.  S.  Department  of  Transportation  issues 
national  curriculum  standards  upon  which 
California  bases  their  curriculum  requirements. 
The  DOT  has  advisory  standing  with  the  states. 

The  California  Office  of  Emergency  Medical 
Services  Authority  is  the  regulatory  agency. 
They  administer  3  programs: 

®  EMT  1  Basic 

•  EMT  2  Intermediate 

•  EMT  3  Paramedic 

EMT  1  &  2  certification  is  acquired  through  an 
approved  training  agency,  usually  Community 
Colleges  or  Junior  Colleges.  EMTs  are 
generally  classified  as  highly  qualified  first  aid 
givers,  but  not  as  medical  technologists.  A 
standard  110  hours  of  instruction  is  required, 
usually  provided  by  Community  Colleges,  in  a 
4  to  5  month  course.  Commercial  schools  may 
also  be  approved.  EMT  1  &  2  may  be 
administered  at  the  County  level,  or  through  an 
association  of  counties  in  less  populated  areas. 
Trainees  are  given  written  and  practical  tests. 
The  County  agencies  can  accept  the  final  exam 
from  an  approved  training  program,  or  they 
may  administer  their  own  tests.  The  California 
State  Fire  Marshal  and  California  Highway 
Patrol  also  administer  EMT  I  programs. 

State  certification,  granted  after  passing  the 
initial  written  and  skill  examination,  is  good  for 
two  years.  Continuing  education  credits,  or  a 
refresher  class,  is  required  to  renew  certification 
each  subsequent  two  year  period. 

Paramedic  certification  (EMT  3  -  Paramedic) 
requires  successful  completion  of  EMT  1  &  2 
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qualification,  plus  1,000  hours  of  required 
training,  usually  provided  by  a  Community 
(Junior)  College.  Persons  with  this  certification 
are  considered  medical  technologists  who  can 
carry  out  specific  medical  practices.  These 
include  intravenous  injections,  and  operation  of 
certain  medical  test  and  life  support  systems. 

State  certification  is  by  initial  written  and 
practical  skill  demonstration  examination  and 
remains  current  for  two  years.  Currency  is 
maintained  by  completing  48  hours  of 
continuing  education  every  two  years,  reported 
to  the  state  board. 

In  order  to  gain  certification,  schools  submit 
their  curriculum  and  qualifications  to  the  State 
for  approval.  Approval  allows  schools  to  be 
included  on  an  approved  list  and  authorizes 
their  programs  for  instruction. 

The  California  Office  of  Emergency  Medical 
Services  Authority  goes  through  a  full  Office  of 
Administrative  Law  process  when  changing 
their  requirements  or  regulations.  There  is  a  45 
day  notice  and  solicitation  of  public  comments, 
then  a  hearing,  etc. 

There  is  a  National  Registry  of  EMTs  and 
Paramedics.  The  National  Registry  is  a  not-for- 
profit,  non-governmental  organization.  It  is 
governed  by  a  Board  of  Directors  made  up  of 
users  of  their  services  and  professional  medical 
people.  They  have  been  in  operation  since 
1970.  They  conduct  certification  and  re¬ 
certification  exams  for  those  states  and 
organizations  who  choose  to  use  them.  They 
conduct  tests  that  some  states  use  for 
certification.  They  feel  they  set  the  standards 
for  the  nation.  They  refer  to  the  DOT  standards, 
but  base  their  standards  on  a  job  analysis. 
Changes  to  the  standards  are  cyclical. 
Sometimes  the  DOT  initiates  a  change  to  which 
they  respond  and  sometimes  technology  or 
technique  improvement  requires  change. 

There  is  also  a  National  Association  of  EMT 
and  Paramedics.  Some  state  and  local 
organizations  provide  forums  and  there  are 
some  private  organizations  that  put  on 
conferences  and  trade  shows. 


American  Sailing  Association  (ASAl 
There  are  no  government  agencies,  including 
the  US  Coast  Guard,  that  require  any  type  of 
certification  for  recreational,  non-commercial, 
water  vessel  operators.  There  is  no 
demonstration  of  skill  necessary  for  commercial 
skippers  operating  water  vessels  under  500  tons 
displacement  under  Coast  Guard  regulations; 
passing  of  a  written  examination  only  meets  the 
certification  requirement.  The  ASA  standard  is 
an  excellent  example  of  a  certification  process 
that  is  maintained  by  a  specific  industry  without 
any  governmental  regulatory  oversight. 

The  American  Sailing  Association  is  dedicated 
to  promoting  safe  recreational  sailing  in  the 
United  States  by  administering  an 
internationally  recognized  educational  system. 
ASA  is  an  association  of  sailors,  professional 
sailing  instructors,  sailing  schools  and  charter 
companies. 

ASA  is  a  private,  for  profit,  organization 
recognized  around  the  world.  Their  association 
with  the  International  Sailing  School 
Association  (ISS  A)  allows  for  recognition  of 
ASA  certification  by  many  national  authorities, 
charter  and  insurance  companies  around  the 
world.  The  group  was  formed  to  promote 
sailboat  operations  safety  and  ensure  acceptable 
levels  of  proficiency  for  various  levels  of 
sailboat  chartering  and  rental. 

Their  Official  International  Log  Book  provides 
information  about  the  standards  and 
certification  requirements  for  various  levels  of 
sailboat  operational  skills  (including  instructor 
certification).  This  group  has  developed  and 
maintains  standards  of  training  and  certification 
for  non-commercial  skippers  who  become 
certified  in  order  to  rent  “bare  boat”  charter 
sailboats  for  pleasure  cruising,  or  various  other 
sail  boats  for  personal  recreation.  The  document 
is  excellent.  It  is  clear,  brief  and  concise  yet 
complete  in  all  essential  details.  The  Log  Book 
is  also  used  to  record  completion  of  the  various 
levels  of  certification.  Review  of  the  Log  Book 
is  required  by  charter  companies  before  a  boat 
is  released  to  a  skipper.  This  system  is  very 
similar  in  nature  to  the  requirements  that  a  pilot 
must  meet  in  order  to  rent  an  aircraft. 
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requirements.  All  standards  are  considered  as 
minimum  for  the  respective  certifications. 

There  are  pre-requisites  for  more  advanced 
certifications.  Starting  with  the  entry  level  in 
the  Basic  Keelboat  Sailing  Standard  that  has  no 
pre-requisites  and  is  described  as:  “Able  to  sail 
a  small  boat  of  about  20  feet  in  length  in  light  to 
moderate  winds  and  sea  conditions  in  familiar 
waters  without  supervision.  A  preparatory 
Standard  with  no  auxiliary  power  or  navigation 
skills  required.”  The  skills  advance  through 
Basic  Coastal  Cruising,  Advanced  Coastal 
Cruising,  though  the  most  advanced  Offshore 
Passage  Making  that  has  the  prerequisites  of  all 
previous  keelboat  and  navigation  standards  and 
is  described  as:  “The  sailor  is  able  to  safely  act 
as  skipper  or  crew  of  a  sailing  vessel  on 
offshore  passages  requiring  celestial 
navigation.” 

All  written  testing  on  “Sailing  Knowledge” 
must  be  passed  with  a  score  of  80%  or  higher 
and  demonstration  of  skill  competency,  “Sailing 
Skill,"  is  evaluated  by  an  ASA  certified 
instructor.  All  certification  is  provisional  until 
reviewed  by  the  organizational  headquarters 
who  issue  the  final  seal  of  approval.  This 
process  is  very  similar  to  FAA  Airman 
certification  as  it  relates  to  their  system  of 
written  testing,  an  oral  and  practical  test 
conducted  by  a  designated  examiner,  followed 
by  review  and  final  certificate  issue. 

Professional  Association  of  Diving  Instruction 
tPADD 

PADI  is  another  example  of  a  non¬ 
governmental  certification  system.  While  not  as 
complex  as  others,  it  serves  the  interest  of 
public  safety  by  ensuring  at  least  basic 
knowledge  before  individuals  may  rent  Self 
Contained  Underwater  Breathing  Apparatus 
(SCUBA)  or  have  air  supply  tanks  filled. 
Approximately  28  hours  of  instruction,  that 
includes  at  least  one  actual  “deep  water  sea 
trial”  (not  in  a  swimming  pool)  dive. 
Certification  includes  both  a  written  test  and 
skills  demonstration  to  the  satisfaction  of  a 
PADI  certified  instructor.  Lack  of  recent 
experience  requires  re-certification  to  assure  the 
diver  remembers  the  safety  factors  and  can 
properly  use  and  operate  SCUBA  equipment. 

The  system  is  very  similar  to  the  one  that  was 
developed  by  ASA  and  has  all  the  basic 


characteristics  of  agencies  that  are  in  the 
standards  and  certification  arena. 

There  is  no  question  that  excellent  models  for 
building  an  organization  to  develop  standards  of 
education,  qualification  and  certification  exist  within 
the  US.  The  organizations  discussed  in  this  chapter 
have  provided  information  freely  and  would  lend 
support  to  others  wishing  to  develop  such  systems.  It 
appears  that  the  aviation  maintenance  industry,  by 
looking  at  the  example  set  by  others  with  similar 
charters  and  interests,  could  move  toward  the 
development  of  a  national  standards  organization 
without  a  high  degree  of  difficulty. 


It  was  not  possible  to  visit  and  meet  with  officials  at 
Transport  Canada  in  Montreal  as  planned.  This 
visit  and  in  depth  discussions,  will  take  place  during 
Phase  II  of  this  project.  There  is,  however,  a  good 
deal  of  information  about  the  Canadian  certification 
system  that  is  pertinent  to  this  phase  of  the  project. 
There  are  aspects  of  the  Canadian  system  that  are 
directly  applicable  to  the  directions  being  taken  in  the 
US  and  may  serve  well  as  a  model. 

The  Canadian  aviation  regulatory  and  certification 
system  is  the  responsibility  of  Transport  Canada  (TC) 
which  is  their  equivalent  of  our  FAA.  While  similar 
to  the  United  States  system  in  many  ways,  there  are 
some  differences  that  should  be  considered: 

®  The  Canadian  aviation  maintenance  industry  is 
smaller  than  that  of  the  USA.  The  current 
number  of  Aircraft  Maintenance  Engineers 
(AME),  who  are  the  equivalent  of  Airframe  and 
Powerplant  Mechanics  (A  &  P),  is  about  32,000, 
versus  about  148,000  A&Ps  in  the  USA. 

»  Transport  Canada  has  recently  revised  the  AME 
certification  process,  moving  more  toward  a 
system  similar  to  the  FAA  system.  This  moved 
Canada  away  from  their  former  system  that  was 
closer  to  their  European  history  and  the 
International  Civil  Aviation  Organization 
(ICAO)  standards  and  practices.  Under  ICAO  all 
maintenance  certification  authority  is  vested  in 
the  Approved  Maintenance  Organization 
(AMO).  An  AME  is  trained  as  a  generalist  with 
specific  aircraft  type-training  requirements, 
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return  to  service  privileges,  and  is  independently 
certified. 

•  There  is  a  group  similar  to  ARAC  in  Canada; 
Canadian  Aviation  Regulatory  Advisory 
Committee  (CARAC)  with  a  working  group  on 
maintenance  certification  and  control.  In 
activities  much  like  those  that  have  been 
conducted  by  the  ARAC  -  65  working  group,  the 
Canadians  are  moving  toward  broader  AME 
licensing  privileges  and  specialist  licenses.  It 
appears  that  their  certification  process  will  move 
even  closer  to  that  of  the  FAA  than  it  is  at 
present. 

•  Apprenticeship  programs  are  in  place  through 
which  an  individual  may  become  certified  as  an 
AME.  These  individuals  are  under  the 
supervision  of  a  qualified  trades  person  learning 
the  principles,  skills,  tools  and  materials  of  the 
trade  while  observing,  practicing  and 
accomplishing  work.  They  also  attend  short 
technical  courses  at  a  college  or  technical 
institute. 

A  1991  Price  Waterhouse  study,  Human  Resources  in 
the  Canadian  Aircraft  Maintenance  Industry, 
sponsored  by  Employment  and  Immigration  Canada 
produced  similar  findings  to  those  of  the  Pilot  and 
Aviation  Maintenance  Technician  Blue  Ribbon 
Panel. 

Canada  has  also  recognized  the  need  for  certified 
specialists  in  specific  skill  areas.  They  have  in  place 
the  Canadian  Aviation  Maintenance  Council 
(CAMC)  which  was  formed  for  the  following 
purposes,  as  stated  in  their  introductory  pamphlet: 

The  council  was  created  to  address  challenges  facing 
the  industry.  These  challenges  were  identified  in  a 
comprehensive  human  resources  study  prepared  for 
the  industry  that  included: 

®  The  need  to  overcome  the  lack  of  formal  training 
programs  available  for  non-licensed  skilled 
tradespersons. 

•  The  need  to  meet  ever  -  rising  requirements  for 
the  entry  into  skilled  trades. 

•  The  need  to  establish  criteria  to  recognize  skills 
of  the  aircraft  maintenance  workers. 


•  The  need  to  increase  retention  of  new  recruits 
especially  among  smaller  employers. 

The  CAMC  is  a  decision-making  body.  It  manages 
current  business,  sets  specific  objectives,  policies  and 
procedures,  and  coordinates  the  efforts  of  various 
committees.  The  committees  cover  topics  such  as 
occupational  standard,  training  programs, 
communications  and  financing,  among  others.  The 
Council  supports  and  encourages  initiatives  to 
develop  the  overall  strength  and  economic  well  being 
of  the  Canadian  Aviation  Maintenance  Industry  both 
locally  and  internationally. 

The  membership  of  the  group  covers  the  full  industry 
spectrum,  represented  by  an  equal  number  of 
employer  and  employee  organizations  including: 

Air  Transport  Association  of  Canada 
Aerospace  Industry  Association  of  Canada 
Canadian  Auto  Workers 
International  Association  of  Machinist  and 
Aerospace  Workers 

Canadian  Federation  of  AME  Associations _ 

CAMC  has  identified  22  occupational  areas  and  is 
currently  developing  occupational  standards  for  these 

thirteen  aviation  maintenance  skills: _ 

Avionics 

Electrical  Component 
Electroplating 

Gas  Turbine  Repair  and  Overhaul 
Interior  Refinishing 
Machinist 

Mechanical  Component 
Non-Destructive  Testing 
Painting 

Reciprocating  Engines  and  Propellers 
Structural  Repair 

Welding _ 

To  ensure  high  quality  standards,  a  technical 
committee,  composed  of  knowledgeable 
tradespersons,  is  established  for  each  skill  area 

(“trade’O. 


Joint  Aviation  Regulations  (JAR)  65,  which  is  the 
European  Economic  Community  (EEC)  equivalent 
to  FAR  Part  65  has  been  in  the  process  of 
development  through  seven  revisions.  It  is  being 
developed  under  the  control  of  the  Joint  Aviation 
Authority  (JAA)  which  is  the  EEC  regulatory  body. 


241 


Training  and  Certification 


Chapter  12 


The  rule  is  not  scheduled  to  become  fully 
implemented  until  July  of  1999.  Harmonization  with 
the  FAR  65/66  is  on  the  agenda,  but  was  not  placed 
on  the  docket  for  1995  /  1996  as  of  the  last  working 
group  meeting  in  March  of  1995. 

The  JAR  65  approach  is  very  different  from  that  of 
both  the  USA  and  Canada,  in  that  all  maintenance 
certification  authority  will  be  vested  in  the  Approved 
Maintenance  Organizations.  It  seems  apparent  that 
JAA  is  committed  to  a  model  that  will  handle 
differences  and  variances  that  exist  between  the 
member  nation  states  through  accommodation.  This 
suggests  that  the  AMO  will  remain  the  basis  for  the 
total  maintenance  certification  control  program. 

Historically,  many  “flag’’  carriers  have  become 
accustomed  to  near  regulatory  control  within  their 
own  country.  These  carriers  seem  hesitant  to  give  up 
this  level  of  influenced  and  control.  The  countries 
that  have  their  own  certification  system  are  not 
comfortable  with  loosing  their  independence  to  a 
system  of  AMO  control. 

Some  countries  place  high  value  and  specific 
requirements  on  structured  formal  training  as  part  of 
certification,  while  others  place  emphasis  on 
certification  based  in  on-the-job  training.  In  some 
cases  maintenance  personnel  are  trainined  to  a  level 
of  qualification  with  no  certification  requirement.  It 
appears  difficult  for  any  consensus  to  be  achieved  in 
this  environment  without  accommodating  many 
divergent  points  of  view. 

The  USA  and  Canada,  who  have  taken  the  approach 
of  centralized  certification  control,  through 
regulating  training,  qualifications  and  certification, 
feel  that  this  is  best  for  all  concerned.  Since  there  is  a 
strong  core  of  agreement  between  the  two  countries, 
and  given  the  recent  North  American  Free  Trade 
Agreement  (NAFTA),  they  are  moving  toward 
harmonization  in  North  America,  which  may  also 
include  Mexico. 

Harmonization  between  JAA  and  the  FAA  may  not 
be  as  simple,  especially  in  the  area  of  maintenance 
technician  certification,  as  initially  thought.  It  also 
appears  that  it  has  become  a  lower  priority  than  it 
was  only  a  few  months  ago.  The  challenges 
presented,  and  differences  that  exist,  between  the 
proposed  JAA  system  and  both  the  US  and  Canada 
do  not  appear  to  be  approaching  resolution  in  the 
near  future. 


Several  professional  organizations  have  been 
suggested  and  /  or  discussed  as  having  potential 
to  become  those  who  may  develop  and  maintain 
aviation  maintenance  advanced  certification 
standards.  It  is  has  also  been  suggested  (for  purposes 
of  harmonization)  that  such  organizations  may  need 
to  be  compliant  with  International  Standards 
Organization  (ISO)  standards  series  9000,  and  /  or  by 
the  Board  of  Accreditation  (RAB)  that  is  part  of  the 
National  Standards  Institute  (NSI).  Following  is  a 
listing  of  possible  organizations: 

Aircraft  Electronics  Association 
American  Society  for  Nondestructive  Testing 
Society  of  Automotive  Engineers 
Air  Transport  Association 
Aircraft  Industry  Association 
Performance  Review  Institute 
National  Aerospace  and  Defense  Contractors 
Accreditation  Program 

Commercial  Aircraft  Composite  Repair  Committee 
(and  several  others  that  may  become  interested) 

There  is  another  point  of  view  that  suggests  that  it 
may  not  be  in  the  national  interest  to  specify  one  or 
more  of  these  existing  organizations  to  hold  the 
“keeper  of  the  flame”  responsibility.  It  may  be  more 
advantageous  to  allow  all  recognized  groups  who 
develop,  validate  and  maintain  standards  to  prepare 
training,  qualification,  and  certification  standards  for 
aviation  maintenance  advanced  skills  as  they  see  fit. 
These  standards,  however,  may  be  required  to 
conform  to  a  set  of  overall  requirements,  developed 
and  maintained  by  a  national  steering,  oversight,  or 
executive  committee.  This  committee,  with 
membership  consisting  of  high  level  industry 
“experts”  would  act  as  the  “keeper  of  the  flame”  and 
endorse  standards  for  aviation  maintenance  advanced 
skills  and  certification.  The  FAA,  in  turn,  would 
accept  certification  standards  that  meet  the  specific 
requirements  of  this  high  level  group  for  ARS-I 
certification.  This  approach  bears  some  similarity  to 
the  CAMC  system  in  Canada,  which  will  be  studied 
further. 

Determination  of  the  industry  and  FAA  views  on  this 
subject  will  be  researched  further;  studied,  reviewed, 
and  reported  upon  in  the  next  phase  of  this  project. 
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During  the  course  of  this  project,  there  are  other 
areas  that  may  be  reviewed  as  having  potential 
for  creating  an  improved  method  for  obtaining 
information  from  the  regulator’s  perspective  and 
input  from  industry,  while  upgrading  industry  / 
government  participation  in  rulemaking. 

Integration  of  ARS  standards  and  FAR  Part  145 

Future  training  scenarios 

AMT  School  self  testing 

FAR  Part  147  flexible  curriculum 

More  privileges  for  AMT  (Annual  inspections 

for  part  91  aircraft,  etc.) 

Harmonization  -  Canada,  NAFTA,  and  rest  of 
the  World 

“Seamless”  maintenance  training  scenarios  from 
primary  through  recurrent. 


The  aircraft  maintenance  industry  is  in  a  state  of 
change.  While  this  state  of  change  has  been  in 
process  over  the  last  decade,  the  rate  of  change  has 
increased  over  the  past  three  to  four  years.  All 
indications  point  toward  the  continuation  of  this 
trend,  at  perhaps  even  a  faster  and  more  dramatic 
rate.  The  regulatory  process,  as  witnessed  by  the  long 
overdue  changes  to  FAR  Part  147  and  the  changes 
currently  in  process  for  FAR  Part  65,  is  slow  to 
respond  and  has  failed  to  keep  pace  with  ongoing 
industry  changes. 

While  the  ARAC  process  may  be  a  starting  point  for 
regulatory  management,  it  needs  to  continue  to 
evolve.  There  is  also  an  apparent  need  to  conduct  a 
more  in  depth  evaluation  of  the  need  to  convene  a 
national  aircraft  maintenance  standards  oversight 
council,  or  committee.  The  membership  may  consist 
of  high  level  aviation  industry  and  FAA  officials  who 
have  strong  process  orientation.  The  group  would 
have  the  “Big  Picture”  of  both  the  technology  and 
maintenance  processes  with  insight  into  how  they 
may  best  be  applied.  It  could  also  serve  as  the 
umbrella  organization  that  provides  oversight  for 
other  groups  that  have  been  qualified  to  issue  and 
maintain  training  and  qualification  standards.  This 
group  could  be  similar  to  the  board  that  has  this  type 
of  function  in  Canada. 


specialists,  advanced  skills  certification  and 
improvement  of  aircraft  maintenance  technician 
professionalism  far  outnumber  the  dissenters.  This 
majority  is  also  cognizant  of  the  need  to  harmonize 
regulations  and  standards,  where  possible,  within  the 
international  community.  They  also  believe  that 
regulatory  congruence  with  Canada  and  other 
NAFTA  countries’  aviation  maintenance  regulations 
will  be  of  significant  benefit  to  North  America  as  we 
move  to  toward  harmonization  with  the  EEC, 
Austral- Asia  and  Middle  Eastern  countries. 


There  seems  to  be  little  doubt  that  a  system  of  this 
type  is  needed.  The  next  phase  of  this  project  will 
more  completely  explore  the  alternatives, 
opportunities  and  necessity  for  development  of 
systems  to  provide  advanced  aviation  technical 
training,  qualification  &  certification.  It  will  provide 
the  foundation  of  information  necessary  to  begin 
putting  the  process  in  place,  and  will  have  established 
the  multi  -  discipline  network  required  to  move 
forward. 


Federal  Register  Part  VI,  Department  of 

Transportation,  Federal  Aviation  Administration, 
14  CFR  Parts  65  and  66,  Revision  of 
Certification  Requirements:  Mechanics  and 
Repairmen;  Proposed  Rule  (Notice  of  Proposed 
Rule  Making:  Notice  No.  94-27) 

Education  and  Training  Standard  for  Entry  Level 
Avionics  (Aviation  Electronics)  Technicians  and 
Engineering  Technologist  Programs.  Association 
for  Avionics  Education  Working  Document, 
Nolan  Coleman,  September  1994. 

ARAC  -  65  Working  Committee  Meeting  Minutes 
and  attendee  notes  from  meetings  conducted 
1992,  1993,  March  3-10,  1994,  May  17-19, 

1994,  July  6-7,  1994,  September  15-17,  1994, 
November  28-30,  1994. 


US  Department  of  Transportation  Federal  Aviation 
Administration  -  Pilot  and  Aviation  Maintenance 
Technicians  for  the  Twenty-First  Century:  An 
Assessment  of  Availability  and  Quality.  Pilot 
and  Aviation  Maintenance  Technician  Blue 


Ribbon  Panel  (Blue  Ribbon  Panel  Report) 
Initiated  by:  Deputy  Associate  Administrator  for 
Regulation  and  Certification,  August  1993 


As  the  study  moves  forward,  support  for  this  type  of 
system  continues  to  grow.  The  supporters  of 
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American  Sailing  Association  -  Official  International 
Log  Book.  Copyright  1993  by  the  American 
Sailing  Association. 

American  Welding  Society  Publications  as  follow: 
OCl-(^  1992  -  Guide  to  AWS  Welding  Inspector 
Qualification  and  Certification 
OC5-G  1992  -  Guide  to  AWS  Welding  Educator 
Qualification  and  Certification 
QC7-93  “  Standard  for  AWS  Certified  Welders 
QC7-93  Supplement  C  -  Welder  Performance 
Qualification  Sheet  Metal  Test  Requirements 
QC7-93  Supplement  F  -  Chemical  Pant  and 
Petroleum  Refinery  Piping 
QC7-93  Supplement  G  -  AWS  Performance 
Qualification  Test 

Employment  and  Immigration  Canada,  Human 

Resources  in  the  Canadian  Aircraft  Maintenance 
Industry:  Price  Waterhouse,  March,  1991. 

Canadian  Aviation  Maintenance  Council, 
Introductory  Pamphlet 

JAR  65  Draft,  NPA  65-0  (dated  1.2.1995) 

APPENDIX  12-A  -  SUBSTANTIVE 
RECQMMENDED  CHANGES  TQ  PART  65 

®  Removal  of  Gender  -Specific  Terms 

®  Re-designation  of  the  Term  "Mechanic" 

•  Equivalency  of  Ratings 

®  Replacement  of  Lost  or  Destroyed  Certificates 

by  Facsimile 


®  Demonstration  of  English-Language  Proficiency 
and  Removal  of  Exception  Criteria  for 
Applicants  Employed  Qutside  the  United  States 
Who  Are  Not  Proficient  in  the  English 
Language. 

•  Establishment  of  a  Requirement  for  Aviation 
Maintenance  Technicians  To  Pass  a  Written  Test 
on  all  Applicable  Provisions  of  Chapter  14. 

®  Clarification  of  Requirement  To  Pass  all 

Sections  of  the  Written  Test  Before  Applying 
for  the  Qral  and  Practical  Tests 

•  Recognition  of  New  Written  Testing  Methods 

®  Specification  of  Experience  Requirements  in 
Hours 

®  Establishment  of  Basic  Competency 
Requirements 

•  Use  of  Equipment-Specific  Training  to  Qualify 
for  Certificate  Privileges 

•  Use  of  Instructional  Time  by  Aviation 
Maintenance  Instructors  to  Satisfy  Currency 
Requirements 

®  Establishment  of  Training  Requirements  for 
Certificated  Aviation  Maintenance  Technicians 
Exercising  the  Privileges  of  their  Certificates  for 
Compensation  or  Hire 

•  Extension  of  Inspection  Authorization  Duration 

•  Expansion  of  Inspection  Authorization  Renewal 
Qptions 
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Ninth  Federal  Aviation  Administration  Meeting  on 
Human  Factors  in  Aircraft  Maintenance  and  Inspection 
Sheraton  Inn  -  Old  Town 
Albuquerque,  New  Mexico 


Agenda 


“HUMAN  FACTORS  GUIDE  FOR 
AVIATION  MAINTENANCE 
AND  INSPECTION” 


Appendix 


Phase  V  Report 


Monday,  November  7, 1994 

3:30  p.m.  -  4:30  p.m.  Registration  -  No  other  activities  planned 

Tuesday,  November  8, 1994 

7:30  a.m.  Continental  Breakfast  and  Registration 

9:00  a.m.  Welcome  and  Meeting  Objectives 
William  T.  Shepherd,  Ph.D. 

Office  of  Aviation  Medicine,  FAA 

9:10  a.m.  Keynote  Address  FAA  Perspective  on  Human  Factors 
Research 

Ms.  Darlene  Freeman 

Associate  Administrator  for  Aviation  Standards,  FAA 

9:30  a.m.  The  Human  Factors  Guide  for  Aviation  Maintenance  - 
Introduction 

Michael  Maddox,  Ph.D. 

Sisyphus  Associates 

10:00  a.m.  Break 

10:20  a.m.  Integrating  Human  Factors  into  a  Maintenance  Program 
Colin  Drury,  Ph.D. 

Dept,  of  Industrial  Engineering,  University  of  Buffalo 

1 1 :00  a.m.  Workplace  Safety 

James  Burnette,  Ph.D. 

Professional  Ergonomist 

1 1:45  a.m.  Lunch  (in  Hotel/Hosted) 

1:15  p.m.  The  "Guide"  -  Demonstration  Exercise 
Michael  Maddox,  Ph.D. 
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Tuesday,  November  8, 1994  cont. 

2:00  p.m.  The  Electronic  Guide  -  A  Demonstration 
Kiki  Widjaja 

Galaxy  Scientific  Corporation 

2:40  p.m.  Human  Factors  in  Maintenance  and  Engineering 
Mr.  Edward  Rogan 

Human  Factors  Manager,  Engineering  Quality  &  Training 

Services 

British  Airways 

3:25  p.m.  Break 

3:45  p.m.  Shift  Work  and  Scheduling 

Ted  Baker,  Ph.D. 

President,  Shift  Work  Systems  Inc. 

4:30  p.m.  Adjourn 

6:30  p.m.  Reception 


Wednesday,  November  9, 1994 

7:30  a.m.  Continental  Breakfast 

8:30  a.m.  The  "Guide"  -  Exercise  Introduction 

Michael  Maddox,  Ph.D. 

8:45  a.m.  Group  Work  on  the  Guide 

10:00  a.m.  Break 
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10:20  a.m.  Group  Reports 

Wednesday,  November  9, 1994  cont. 

1 1 :30  a.m.  Human  Factors  Guide  Evaluation/Discussion 
Michael  Maddox,  Ph.D. 

1 1 :45  a.m.  Lunch  (in  Hotel/Hosted) 

1 :00  p.m.  Performance  Enhancement  System  (PENS):  Update 
Charles  Layton,  Ph.D. 

Galaxy  Scientific  Corporation 

1 :30  p.m.  CD-ROM  #2  -  A  Demonstration 
William  Johnson,  Ph.D. 

Galaxy  Scientific  Corporation 

1 :50  p.m.  Sandia  NDI  Research  Overview 
Patrick  Walter,  Ph.D. 

NDI  Validation  Facility,  Sandia  Laboratory 

2:20  p.m.  Closing  Comments 

William  T.  Shepherd,  Ph.D. 

Office  of  Aviation  Medicine,  FAA 

2:30  p.m.  Bus  trip  to  Aging  Aircraft  NDI  Validation  Facility 

3:00  p.m.  NDI  Center  Tours 

4:30  p.m.  Transit  to  Hotel 

5:00  p.m.  Adjourn 
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Human  Factors 
In  Aviation  Maintenance 


Dr.  Jon  L  Jordan, 
Federal  Air  Surgeon 


Aviation  safety  is  dependent  on  the  cooperation  of 
operators,  manufacturers,  and  the  government 
(FAA).  That  triad  is  often  referred  to  as  “the  three 
legged  stool,"  with  each  leg  representing  one  of  these 
groups.  I’m  sure  all  of  you  have  seen  that  stool 
depicted  before  in  other  seminars  and  workshops. 

The  stool  must  have  all  three  legs  to  be  balanced  and 
safe. 

The  Office  of  Aviation  Medicine  Human  Factors  in 
Maintenance  Research  Program  bolsters  the 
association  and  “the  working  together  attitude”  of 
government,  manufacturers,  and  operators.  Aviation 
Medicine  does  that  through  conferences  like  this  one, 
through  extensive  publications,  by  distribution  of  CD- 
ROMs  containing  data  and  information,  and  from 
active  participation  in  meetings  worldwide. 

Aviation  Standards  is  proud  of  the  6  year  history  of 
the  Human  Factors  in  Aviation  Maintenance  Research 
program.  As  Associate  Administrator,  I  appreciate 
the  opportunity  to  be  here,  to  speak  with  you,  and  to 
see  first-hand  why  these  Human  Factors  Workshops 
have  become  so  popular. 

The  historical  basis  for  embarking  on  this  research 
program  can,  most  likely,  be  attributed  to  the  1988 
Aloha  accident.  The  accident  showed  that  human 
performance  issues,  are  critical  componets  for  safety, 
no  different  from  adherence  to  Service  Bulletins,  use 
of  proper  inspection  procedures,  and  maintenance 
training.  The  stark  picture  of  the  wrecked  737  moved 
Congress  to  quickly  pass  legislation  that  was  already 
under  consideration  at  the  time.  The  Aviation  Safety 
Act  of  1988  specified  that  the  Department  of 
Transportation  shall  conduct  safety  research,  which 
must  attend  to  “human  performance”  not  only  in  the 
cockpit  but  also  in  maintenance  operations.  That 
legislation  was  significant.  It  gave  us  the  financial 
wherewithal  to  focus  research  on  the  aviation 
maintenance  technician  as  the  central  and  most 
critical  part  of  the  total  aviation  maintenance  system. 


The  studies  we  have  performed  since  1988  have 
clearly  shown  that  human  performance  in 
maintenance  can  have  a  strong  impact  on  safety. 

In  1991  the  FAA  published  the  National  Plan  for 
Aviation  Human  Factors.  That  plan,  authored  by  a 
panel  of  50  human  factors  professionals  from 
industry,  academia,  and  government,  also  emphasized 
the  need  for  maintenance  research. 

Recommendations  from  the  National  Plan  have 
guided  the  Aviation  Medicine  research,  but  just  as 
importantly,  feedback  you  have  given  us  at  these 
workshops  has  shaped  our  program. 

While  safety  is  always  our  foremost  concern  at  FAA, 
we  cannot  ignore  the  current  tough  economic  times  in 
the  air  transportation  industry.  We  must  work  with 
industry  to  be  sure  that  safety  is  never  compromised 
by  economics.  This  research  program  has  been 
successful  partly  because  of  its  attention  to  economics 
and  to  providing  cost-effective  research  solutions  and 
products  that  enhance  human  performance.  The 
results,  therefore,  are  improvements  in  efficiency 
while  maintaining  the  high  safety  standards  of  our 
industry. 

Six  years  of  research,  and  now  nine  workshops,  have 
already  created  a  research  legacy.  I  would  like  to 
briefly  highlight  a  bit  of  history  and  a  few  of  the 
milestones  that  have  been  achieved  in  our  research 
program,  and  the  contributions  made  by  you  and  your 
colleagues  in  maintenance  and  in  human  factors 
research. 

The  first  workshop,  was  held  in  the  Fall  of  1988.  At 
that  time  we  were  formatting  our  research  plans  and 
establishing  key  industry  liaisons.  When  I  say  “we”  I 
mean  not  just  FAA  but  all  of  us  who  have  been 
working  on  maintenance  human  factors  including  you 
folks  from  industry,  academia  and  other  government 
agencies.  I  am  really  happy  to  say  that  “we”  have 
been  an  effective  team  that  has  produced  some  solid 
results  in  the  past  few  years.  Please  understand  that 
very  few  of  these  results  would  have  been  achieved 
without  your  active  participation. 
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The  topic  of  Human  Factors  in  Maintenance  and 
Inspection  was  a  new  one  at  the  time  of  the  first 
meeting.  While  some  maintenance  research  had  been 
done  in  the  US  Military,  Aviation  Human  Factors 
research  was  primarily  a  study  of  cockpit  and  air 
traffic  control  issues.  Research  in  maintenance 
human  factors  was  almost  non-existent.  A  few  key 
problems  were  identified  at  that  first  meeting.  The 
topic  that  received  the  most  attention  was 
“communication,”  including  just  about  every 
imaginable  communication  form  ranging  from  design 
of  job  cards  to  use  of  understandable  language  in 
manuals,  to  holding  workshops  like  this  one.  It  was 
suggested  at  the  first  workshop  that  we  have  “at  least 
one  additional  meeting  of  this  kind....”.  This  being 
the  ninth  meeting  -  you  certainly  followed  that 
recommendation!  Meeting  attendance  has  increased 
300%,  and  would  probably  be  greater  if  we  did  not 
limit  the  number  of  workshop  participants. 

In  six  years  the  research  team  has  published  over  150 
articles  and  conference  papers.  For  four  years  now 
the  prestigious  Human  Factors  and  Ergonomics 
Society  has  dedicated  a  complete  session  to  the 
Aviation  Medicine  Maintenance  Human  Factors 
program.  We  presented  again  this  Fall  at  their 
meeting  in  Nashville.  We  have  supported  the 
International  Civil  Aviation  Organization  (ICAO),  by 
lecturing  at  9  Regional  Seminars  on  Human  Factors 
in  Aviation  Maintenance.  The  research  team  has 
participated  in  a  variety  of  Air  Transport  Association 
and  FAA  meetings  as  well. 

That  first  meeting  produced  some  other 
recommendations  about  maintenance  training,  or  the 
lack  thereof,  and  FAA  has  paid  attention  to  those 
recommendations.  First,  the  Flight  Standards  Service 
modified  FAR  Part  147  dealing  with  curriculum  in 
Aviation  Maintenance  Technicians  Schools.  Also, 
FAA  established  a  Blue  Ribbon  Panel  on  the 
Shortage  of  Personnel  for  Aviation  Occupations. 

Data  from  this  human  factors  research  program 
played  a  role  in  the  conclusions  and  recommendations 
of  the  Blue  Ribbon  Panel.  Likewise  the  panel 
provided  ideas  that  have  become  topics  of  current 
human  factors  research.  Our  current  study  related  to 
innovative  maintenance  certification  practices  is  one 
such  research  example. 

The  program  has  conducted  research  ranging  from 
basic  scientific  studies  at  the  University  level  and  at 
the  Civil  Aeromedical  Institute,  to  applied 


development  and  evaluation  in  the  airlines  and 
schools. 

In  1989  the  team  was  introducing  such  terms  as 
“intelligent  tutoring,  and  expert  system  trainers”  for 
aviation  maintenance.  You  built,  evaluated,  and 
distributed  such  systems  at  training  schools  and  with 
cooperating  airlines.  Much  of  the  “advanced 
technology  training”  language  this  research  team  was 
innovating  in  1989  has  become  common  training 
language  used  today. 

The  1988  meeting  also  placed  importance  on 
maintenance  information.  Your  program  has  studied 
information  collection,  format,  storage,  retrieval  and 
display.  You  have  worked  with  airlines  to  redesign 
and  evaluate  job  work  cards.  You  have  assisted  the 
Flight  Standards  Service  with  design,  development, 
and  fielding  of  the  pen  computer-based  Performance 
Enhancement  System,  or  “PENS”  as  it’s  known 
today.  That  work  is  continuing  with  a  look  at  other 
types  of  electronic  support  systems  both  for  airlines 
and  FAA  inspectors. 

Your  systems’  analytic  approach  to  the  design  and 
fielding  of  PENS  has  set  an  exemplary  standard  for 
new  technology  design  development  and 
implementation.  The  concept  of  “human-centered” 
design  and  extensive  use  of  rapid  prototypes  have 
become  very  popular  with  users  and  with  FAA 
management.  Unequivocally,  PENS  is  enhancing  the 
work  performance  of  Aviation  Safety  Inspectors. 

With  respect  to  information  technologies,  your 
program  was  the  first  to  publish  and  distribute 
information  on  CD-ROM.  CD-ROMs  represent  the 
cutting  edge  of  information  technologies.  There  is  no 
question  that  they,  and  similar  systems  will  be  critical 
to  the  efficiency  and  quality  of  future  Aviation 
Maintenance  operations,  not  to  mention  the 
performance  of  our  own  FAA  oversight  activities. 

While  the  Aviation  Maintenance  research  program 
has  compiled  a  good  track  record,  it  is  you  the  people 
in  this  audience  and  your  colleagues  who  deserve  the 
credit.  This  research  program  is  applied  -  it  is  in 
tune  with  industry  conditions  and  requirements.  We 
feel  that  the  industry  has  welcomed  our  input  and  has 
been  very  candid  in  reporting  many  opportunities  for 
improvement. 

The  list  of  airlines  cooperating  with  our  research 
program,  is  pretty  impressive.  Atlantic  Southeast, 
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Continental,  Delta,  Northwest,  US  Air,  United, 
Southwest,  TWA,  British  Airways,  and  Lufthansa 
have  all  taken  an  active  role  as  host  sites  for  data 
collection  or  as  speakers  at  these  workshops. 

Numerous  others  have  permitted  on-site  operations  to 
allow  observation  and  measurement,  and  provided 
personnel  time  and  advice.  Manufacturers  have  also 
been  generous  with  their  time  and  counsel. 

The  single  most  important  characteristic  of  the 
research  program  is  its  attention  to  “real-world” 
aviation  maintenance  considerations.  Again  our 
thanks  to  you  -  in  industry  --  for  your  help. 

So  much  for  the  past;  What  about  the  Future  of 
Maintenance  Human  Factors  Research? 

Have  all  the  human  performance  issues  been 
addressed?  Is  the  system  “fixed?”  Is  the  work  of 
this  research  program  over?  The  answer  to  all  of 
these  is  “No.”  There  are  still  plenty  of  untouched 
opportunities  to  enhance  human  performance  in 
maintenance. 

As  current  or  older  technology  aircraft  continue  to  fly 
there  must  be  increasing  maintenance  diligence  and 
caution  to  ensure  continuing  airworthiness.  These 
aircraft  will  require  new  maintenance  practices  and 
inspection  procedures.  These  procedures  will  have  to 
capitalize  on  new  technology  methods,  which  will 
come  about  through  research. 

New  aircraft  will  require  new  skills.  Technicians 
already  accustomed  to  aircraft  built-in-test  equipment 
will  have  to  convey  this  knowledge  to  their  co¬ 
workers.  Changes  in  FAR  Part  147,  and  plans  for 
changes  to  Part  65,  already  have  placed  emphasis  on 
new  electronics-oriented  skills  for  the  aviation 
maintenance  technician.  In  fact  we  see  this  as  one  of 
the  most  important  areas  for  our  research.  Terms 
such  as  “the  information  super  highway”  are  familiar 
to  all  of  you.  These  terms  are  especially  pertinent  in 
aircraft  maintenance  where  it  is  crucial  that  proper 
information  is  exchanged.  The  volume  of  data  and 
information  needed  to  maintain  today’s  air  carrier 
fleet  is  even  now  approaching  overwhelming 
proportions  and  it  will  become  even  larger  in  the 
future.  If  technicians  are  not  to  spend  all  of  their  time 
looking  up  or  finding  repair  information,  they  must 
have  assistance,  electronic  assistance,  in  getting  the 
information  needed  to  perform  their  jobs. 


technical  data,  available  to  the  technician.  Such  data 
must  be  provided  at  the  aircraft.  Otherwise, 
technicians  would  spend  too  much  time  traveling 
around  obtaining  needed  information.  Portable 
maintenance  access  terminals,  hand-held  computers, 
and  other  such  technologies  promise  to  change  the 
way  technicians  work.  I  think  we  are  at  the  threshold 
of  some  dramatic  changes  in  the  way  maintenance 
will  be  performed  in  the  future.  For  the  air  carrier 
industry  to  stay  safe,  competitive  and  viable  there  will 
have  to  be  some  big  improvements  in  efficiency  of 
their  operations  across  the  board,  but  particularly  in 
the  realm  of  maintenance.  We  want  our  research  to 
be  on  the  forefront  of  this  transition  to  help  provide 
industry  with  the  knowledge  and  tools  needed  to 
make  their  operations  as  safe  as  possible  and  also 
achieve  new  efficiencies. 

The  work  of  human  factors  researchers,  obviously, 
must  position  the  industry  to  maximize  the 
performance  of  humans  in  the  maintenance  system. 
That  job  is  never  completed.  The  good  news  is  that 
the  FA  A  and  Industry  recognize  the  importance  of  the 
investment  in  human  performance  research  and 
development.  We  are  on  the  right  path  and  we  have 
participation,  and  great  support  from  government, 
industry,  operators,  and  academia.  We  must 
continue,  on  course,  with  the  Aviation  Medicine 
Human  Factors  in  Maintenance  research  and  other 
related  endeavors. 

For  the  next  day  and  a  half  you’ll  be  discussing  a 
draft  version  of  the  Maintenance  Human  Factors 
Handbook.  This  handbook  has  been  in  preparation 
for  the  last  few  years  and  depends  quite  a  bit  on 
results  obtained  from  the  research  program.  We  want 
the  handbook  to  be  a  useful  document  for 
maintenance  management.  Typical  research  reports 
in  scholarly  journals  are  not  of  much  use  to  people 
who  work  on  the  front  lines  in  industry.  These  people 
have  real  world  human  factors  problems  they  have  to 
deal  with  every  day  and  we’ve  tried  to  aim  our 
handbook  at  those  types  of  problems.  We  want  our 
handbook  to  be  used,  so  during  this  meeting  you're 
going  to  practice  with  some  hypothetical  human 
factors  scenarios  that  might  be  found  in  a 
maintenance  organization  and  show  how  the  book  can 
be  put  to  good  use.  I  hope  the  next  couple  of  days 
will  be  rewarding  and  informative. 

Again,  I  appreciate  the  opportunity  to  participate  in 
this  meeting  with  you.  I  look  forward  to  continued 
involvement  with  the  program. 


Technical  information  and  the  way  it’s  provided  are 
changing.  There  will  be  ah  increasing  volume  of 
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Human  Factors 

IN  Maintenance  and  Engineering 


Mr.  Edward  Rogan 
Novembers,  1994 


Good  afternoon.  Ladies  and  Gentlemen.  It  really  is 
a  pleasure  to  be  here  and  I  would  like  to  thank 
Bill  for  inviting  me  along  to  address  the  conference. 
As  Bill  said  I  am  the  Human  Factors  Manager 
Engineering  at  British  Airways  which  is  a  fairly 
unique  beast  in  our  industry.  The  actual  position  was 
created  in  March  of  this  year,  so  I’m  a  bit  of  a  new 
kid  on  the  block.  Also  from  my  introduction,  you  can 
see  that  career  is  all  within  aircraft  engineering  and 
maintenance  and  I  have  no  psychology  or  Human 
Factors  background.  However,  I  am  fortunate  enough 
to  have  a  Human  Factors  Consultant  working  with 
me  who  is  very  supportive  in  those  areas  engineers 
are  traditionally  very  bad,  you  know,  things  like 
spelling,  grammar  and  punctuation  to  mention  but  a 
few.  Again  from  my  introduction  you  would  have 
seen  my  background  has  been  primarily  in  what  we 
call  Technical  Services.  I  think  in  the  States  you 
would  call  it  Engineering.  In  that  time  as  a 
development  engineer,  I  was  continually  frustrated 
when  involved  in  a  quality  lapse  investigation  to 
conclude  that  the  cause  of  the  incident  was  human 
error.  I  was  therefore  very  pleased  that  Human 
Factors  gave  us  the  opportunity  to  go  deeper  into 
these  investigations  and  look  for  the  actual  cause  of 
the  quality  lapse  not  the  consequence.  So  this  was 
one  of  the  reasons  why  I  was  attracted  to  the  subject 
of  Human  Factors.  I  believe  the  appointment  of  a 
Human  Factors  Manager  Engineering  shows  a  lot  of 
commitment  to  the  subject  by  British  Airways,  our 
task  now  is  turning  that  commitment  into  action.  One 
thing  I  should  mention  is  that  we  believe  not  one 
person  can  have  complete  responsibility  for  Human 
Factors,  we  see  everyone  in  the  organisation  as 
having  this  responsibility.  We  see  Human  Factors  in 
terms  of  organisational  culture  where  everybody 
understands  and  has  bought  into  it  as  well  as  owning 
it,  very  much  like  the  current  Total  Quality  ethos  at 
BA. 

So  what  am  I  going  to  talk  about  today?  I  would  like 
to  give  a  brief  overview  of  some  of  the  British 


Airways  Human  Factors  initiatives  with  which  we  are 
currently  involved  or  planning  to  do  in  the  future.  I 
would  then  like  to  concentrate  on  one  particular 
initiative  called  MESH.  I  will  describe  what  MESH 
is,  where  it  came  from  and  what  it  looks  like,  which 
means  another  computer  simulation  or 
demonstration.  I’ll  talk  a  bit  about  the  trial  we  have 
ongoing  at  the  moment  in  British  Airways.  I’ll  just 
share  some  very  early  results  and  I  wouldn’t  like 
anyone  to  read  too  much  into  these  results.  Finally,  I 
would  like  to  talk  about  where  we’re  going  with 
MESH  in  the  future  and  then  take  any  questions  you 
may  have  for  me. 

So  what  Human  Factors  initiatives  do  we  have  at 
British  Airways  Engineering?  First  and  most 
important  of  all,  we  need  commitment  from  the  top 
before  we  can  change  the  culture  of  an  organisation. 
It’s  a  top-down  approach.  We  have  a  Human  Factors 
Steering  Group  which  meets  every  two  months  and 
has  done  for  the  last  two  years  and  is  attended  by  our 
General  Managers.  This  high  level  involvement 
ripples  through  the  organisation  to  say  that  we  have 
commitment  here  and  that  is  essential  for  others  to 
see  before  they  buy-in. 

As  I  said  before,  we  see  an  organisational  culture 
change  as  a  key  element  in  any  overall  Human 
Factors  programme.  To  change  culture  you  must  first 
make  everyone  aware  of  the  subject  and  change  some 
traditional  attitudes  towards  human  error. 

We  have  started  to  produce  a  quarterly  Human 
Factors  magazine  with  a  circulation  of  about  fifteen 
hundred,  the  third  issue  is  due  to  be  published  next 
month.  These  are  distributed  to  staff  in  our  Hangars, 
workshops  and  offices  throughout  Engineering.  The 
magazine  not  only  allows  staff  to  learn  from  past 
mistakes  and  incidents  but  also  make  them  more 
aware  of  the  subject  by  introducing  them  to  some  of 
Prof.  Jim  Reason’s  (University  of  Manchester) 
theories  on  human  error.  The  magazine  also  serves  as 
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a  way  of  communicating  the  progress  of  some  of  the 
other  Human  Factors  initiatives  which  1  will  be 
talking  about  later. 

I  would  like  to  now  talk  about  Human  Factors 
education.  We  are  currently  involved  with  an 
education  programme  for  managers  which  we  call 
Emergent  Leadership.  This  intensive  programme 
encourages  managers  to  re-examine  many  of  their 
values  (e.g.,  blame  and  fear  free  culture,  openness 
and  honesty,  etc.)  in  order  to  change  traditional 
attitudes  and  behaviour  which  should  then  lead  to, 
what  we  would  call,  a  Human  Factors  culture  in  the 
organisation.  We  are  cun'ently  looking  at  a  future 
programme  for  all  grades  similar  to  the  CRM  courses 
for  Fight  crew. 

On  Quality  Lapse  investigations,  we  are  working 
with  Boeing  on  a  process  and  database  known  as 
MEDA,  Maintenance  Error  Decision  Aid.  This  is 
basically  a  process  to  actually  discover  the  root 
causes,  and  take  away  some  of  the  blame  from  the 
individuals  involved  in  quality  lapses.  The  process  is 
currently  paper-based,  however,  the  plan  is  to 
develop  a  world-wide  computer  database  of 
maintenance  error  involving  many  airlines  .  We  see 
this  as  a  worthwhile  project  with  which  to  be 
involved. 

And  finally  MESH  (Managing  Engineering  Safely 
Health),  which  we  refer  to  as  a  proactive  Human 
Factors  tool  as  opposed  to  MEDA,  which  we  term 
reactive  because  this  is  event-driven,  i.e.,  an  incident 
needs  to  have  occurred  before  you  can  investigate  it. 
MESH  was  conceived  by  Prol'essor  Jim  Reason  and 


his  team  from  the  University  of  Manchester.  The 
system  is  an  anonymous  human  factors  computer- 
based  evaluation  or,  if  you  prefer,  a  survey  of  staffs 
perceptions  to  Human  Factors  in  their  area.  It  is 
important  to  state  that  MESH  is  not  a  reporting  tool 
and  it  is  not  intended  to  replace  our  current  reporting 
processes.  MESH  is  a  proactive  safety  management 
tool,  the  results  from  which  can  be  used  to  identify 
and  prioritise  the  human  factors  issues  in  a  particular 
area.  It’s  not,  however,  just  a  management  tool.  With 
the  Total  Quality  culture  that  we  have  at  BA 
Engineering,  we  see  it  very  much  as  everyone  getting 
involved  and  resolving  their  own  local  issues.  Of 
course,  there  will  be  large,  long  term  issues  that 
affect  the  whole  organisation  which  cannot  be 
addressed  locally.  We  must  therefore  also  analyse  the 
results  from  an  organisational  point  of  view. 

MESH  looks  at  two  types  of  factors,  which  we  term 
Local  and  Organisational  Factors.  Local  Factors,  or 
LFs,  are  those  factors  that  affect  staff  at  the  sharp  end 
of  the  operation.  These  factors  are  actually  specific  to 
the  area  and  have  been  selected  by  staff  in  that  work 
area;  typical  examples  of  LFs  are  things  like  Parts 
Availability,  Fatigue,  Tools  and  Equipment, 
Knowledge,  Skills  and  Experience,  etc. 

Organisational  Factors,  or  OFs,  are  constant 
throughout  the  engineering  organisation,  and 
although  these  affect  everyone  in  the  organisation, 
they  arc  more  visible  to  management.  Organisational 
Factors  are  therefore  rated  by  management  grades; 
typical  examples  of  OFs  are  training  and  selection, 
people  management,  provisions  of  tools  and 
equipment,  building  and  equipment  maintenance,  etc. 


ianagSoig  Engineering  Safety  Heagth” 


Based  on  theory  conceived  by  Prof.  Jim  Reason  (Manchester  University,  England) 
Anonymous  Human  Factors  computer  based  evaluation 
Proactive  survey  of  staff’s  perceptions  of  Human  Factors  in  their  workplace 
Results  used  to  identify  and  prioritise  Human  Factors  issues  in  the  workplace _ 


Local  and  Orginisational  Factors 
o  Local  Factors  (LFs): 

o  Affect  those  at  the  sharp  end. 

®  Specific  to  an  area  and  selected  by  users. 

©  Eg.,  Parts  Availablility  and  Fatigue. 

Organisational  Factors  (OFs): 

o  Affect  Everyone,  but  more  visible  to  management. 

©  Standard  questions  throughout  organisation. 

©  Eg.,  Training  and  Selection  and  People  Management. 
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I  would  like  to  now  give  a  demonstration  of  the 
MESH  software.  Here’s  a  scenario.  One  week  ago,  a 
licensed  avionics  engineer  waited  for  seven  hours  for 
a  spare  part.  The  aircraft  departed  on  time,  but  the 
installation  of  the  part  and  the  subsequent  function 
tests  needed  to  be  done  extremely  quickly.  The 
licensed  aircraft  engineer  is  rostered  to  do  his  MESH 
input  today.  You  can  see  MESH  has  been  designed  to 
be  used  in  the  Windows  environment.  It  is  extremely 
user  friendly.  Not  all  of  our  technicians  and  aircraft 
engineers  are  actually  PC  literate,  so  we  wanted  to 
make  it  as  simple  as  possible,  this  was  an  important 
specification  for  the  system.  Using  the  mouse,  the 
licensed  aircraft  engineer  would  click  on  the  MESH 
icon.  The  first  question  I’m  asked  is  my  Grade.  Okay, 
I’m  a  Licensed  Aircraft  Engineer.  All  the  grades  in 
the  list  are  particular  to  that  area,  so  as  I  said  before 
MESH  is  defined  and  designed  for  that  area.  The 
next  question  I  am  asked  is  my  trade  -  in  my  case 
Avionics.  The  final  personal  question  is  what  shift 
am  I  on  -  let’s  say  A  shift.  Again  the  shifts  are  all 
applicable  to  this  one  area.  I’ve  used  it  though,  I 
would  like  some  more  questions.  The  next  screen 
also  ask  several  questions  but  this  time  they  are 
related  to  the  task.  The  system  first  asks  you  to  think 
of  a  task  in  the  last  week  that  did  not  go  to  plan  and 
which  may  have  caused  frustration  or  put  pressure  on 
the  individual  to  complete.  It  then  asks  for  the  Date  - 
last  week.  ATA  chapter  -  instruments.  Airline 
operator  -  BA  in  this  case.  Source  of  work  -  let's  say 
it  was  an  AMS  item. 

Now,  because  I  have  told  the  system  that  I  am  an 
aircraft  engineer,  the  Local  Factors  evaluation 
appears.  If  I  had  said  that  I  was  a  line  manager  then 
the  Organisational  Factors  would  have  appeared.  The 
first  factor  I  am  asked  to  rate  is  Knowledge,  Skills 
and  Experience.  But  what  does  this  mean  to  me?  By 
clicking  on  the  example  box  next  to  the  factor  gives 
typical  examples.  These  examples  have  been  again 
derived  by  staff  in  that  area  and  are  therefore 
normally  pertinent  to  typical  tasks  performed  in  that 
area.  In  this  case  Knowledge  Skills  and  Experience 
wasn’t  a  problem,  so  I  click  on  No  Problem.  The  next 
factors  Morale,  Fatigue,  Time  of  Day,  Manuals  and 
Procedures  and  Tools  and  Equipment  were  not  a 
problem  in  my  example,  so  again  I  click  on  No 
Problem.  Parts  and  Spares  -  well,  this  was  my 
problem  and  by  clicking  on  the  example  box,  the 
system  prompts  me  to  think  whether  it  was  a  quality, 
location,  availability  or  delivery  problem.  In  my  case 
it  was  the  time  taken  for  delivery  of  the  part.  So  for 
Parts  I  would  rate  this  as  quite  a  problem.  The 


remaining  factors  were  also  not  a  problem  in  my  case 
except  maybe  operational  pressure  which  I  will  rate 
as  a  Nuisance.  So  the  next  screen  that  appears  is  the 
Anonymous  Comment  facility.  The  initial  version  of 
MESH  didn’t  have  this.  This  facility  makes  it  easier 
to  determine  the  real  issues  behind  the  MESH 
Profiles  when  these  are  analysed  at  a  later  date.  In 
our  case,  you  would  normally  put  in  something  like  - 
waited  for  seven  hours  for  a  NAV  receiver,  Pt  No. 

123 ABC  from  SPUD  stores.  That’s  probably  BA 
speak,  but  you  would  try  and  be  specific  as  you  can 
and  the  information  is  then  stored  in  the  data  base 
along  with  all  the  other  comments  from  that  area. 

Now,  it’s  important  that  feedback  is  given  to  the  user, 
so  we  have  the  facility  here  that  not  only  shows  the 
Local  Factors  Profile  for  the  individual’s  input  but 
also  for  the  whole  area  over  the  last  month.  I  can  also 
look  at  the  Organisational  Factor  Profile  for  the 
month.  I  hope  I  have  shown  you  how  easy  and  quick 
the  system  is  to  use.  There  is,  however,  a  Help 
facility  for  those  who  have  forgotten  how  to  use  the 
system.  The  Help  screens  also  give  information 
regarding  human  error  theory,  again  this  has  been 
based  Prof  Reason's  work. 

I  am  now  going  to  click  on  the  MESH  Analysis  icon. 
This  tool  is  used  by  whoever  is  responsible  for 
looking  at  the  profiles  and  identifying  the  issues 
behind  them.  We  plan  to  modify  this  part  of  the 
software  over  the  next  few  months,  however,  I  will 
demonstrate  the  software  that  we  have  been  using 
over  the  last  year.  The  Analysis  tool  gives  us  the 
ability  to  sort  the  collected  MESH  data  in  order  that 
we  may  compare  different  time  periods,  work  areas, 
shifts,  trades  etc.  We  can  then  use  these  results  for 
trending  purposes.  Okay,  let  me  exit  the  analysis  tool 
and  I'd  like  to  now  talk  about  the  actual  MESH  trial 
at  BA  Engineering. 

MESH  is  currently  installed  in  five  work  areas;  two 
hangars,  one  workshop  and  two  engineering  offices. 
These  are  very  different  environments  and  this  is  one 
of  the  reasons  for  allowing  the  local  area  to  define  its 
own  requirements  for  the  system;  what  works  in 
Hangar  X  probably  will  not  work  in  Office  Y.  We 
have  two  MESH  Champions  per  work  area  who  train 
staff  in  the  area  on  how  to  use  the  system.  They  also 
analyse  MESH  data  from  their  area  using  the  MESH 
analysis  tool  which  I  have  just  demonstrated. 
Existing  computer  hardware  has  been  utilised  in  all 
these  areas.  More  and  more  PCs  are  appearing  in  the 
hangars  and  other  areas  and  therefore  there  shouldn’t 
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be  any  hardware  costs  associated  with  installing 
MESH.  We  have  a  monthly  MESH  Progress  meeting 
to  coordinate  the  trial,  which  is  attended  by  all  the 
champions,  myself,  the  human  factors  consultant  and 
my  General  Manager. 

The  local  process  for  managing  MESH  data  is  left  to 
the  local  areas  to  develop,  although,  I  do  give  a 
number  of  guidelines  based  on  our  experience  to 
date;  such  as  target  number  of  MESH  inputs  per 
month.  How  they  achieve  this  is  very  much  up  to 
them,  because  as  I  said  before,  all  the  environments 
are  different  and  they  know  best  how  their  staff  need 
to  be  encouraged  to  use  the  system.  For  hangar  areas, 
we  found  it  best  to  go  for, a  twenty  percent  sample. 
We  would,  therefore,  ask  a  random  sample  of  twenty 
percent  of  the  workforce  to  enter  data  once  a  month 
for  three  months,  and  then  they  wouldn't  have  to 
enter  data  for  another  year.  Then,  there  would  be 
another  group  of  staff  who  would  do  it  for  three 
months,  and  so  it  goes  on.  In  areas  where  there  isn't 
large  numbers  of  staff  employed,  we  normally  ask 
everyone  to  input  once  a  month  in  order  to  achieve  a 
significant  sample. 

I  mentioned  before  that  we  don’t  see  this  as  just  a 
management  tool  and  we  actively  encourage  local 
TQ  working  groups  to  use  MESH  results  in  order  to 
resolve  Human  Factors  issues  in  their  area.  In  some 
areas,  the  monthly  MESH  profiles  are  on  the  agenda 
of  existing  local  TQ  forums.  MESH  profiles  make  a 
very  good  agenda  item,  identifying  and  prioritising 
issues  to  talk  about  and,  most  importantly,  resolve.  I 
think  it  is  important  that  we  don’t  identify  hundreds 
of  issues  and  never  get  around  to  resolving  any  of 
them.  MESH  allows  us  to  prioritise  our  efforts, 
concentrating  on  resolving  maybe  two  or  three  issues 
identified  as  contributing  to  the  high  bar  of  the 
MESH  profile.  Feedback  to  MESH  users  is  very, 
very  important  in  order  to  motivate  staff  to  use  the 


system.  Actions  taken  and  resolutions  must  regularly 
fed  back  and  even  if  you  can’t  take  any  action,  it  is 
important  to  say  why  not.  If  used  correctly,  MESH 
should  help  communication  within  an  area.  For 
example,  MESH  has  identified  a  deep  rooted 
problem  which  will  take  the  organisation  two  years  to 
resolve.  By  feeding  back  to  staff  information  on  what 
the  organisation  is  planning  to  do  about  the  problem 
and  when  can  influence  the  attitude  of  staff  and  have 
a  positive  influence  on  the  MESH  profiles.  The 
method  of  feedback  varies  according  to  the  area,  but 
is  generally  given  in  either  newsletters,  notice  boards 
or  the  human  factors  magazine.  We  are  considering 
giving  feedback  electronically,  by  having  another 
icon  in  the  MESH  box  saying,  for  example,  MESH 
Feedback,  so  that  a  monthly  update  on  MESH  issues 
can  be  accessed  by  staff  using  the  MESH  PC. 

Another  important  use  of  MESH  Profiles  is  looking 
at  the  effectiveness  of  past  resolutions.  Profiles  can 
be  used  to  support  business  cases  to  show  not  only 
that  a  problem  exists,  but  also  we  now  have  a  way  of 
monitoring  in  the  future  the  effectiveness  of  the 
resolution  proposed  in  the  business  case. 

Now,  I  have  stated  that  MESH  is  basically  a  local 
tool,  but  obviously  there  are  going  to  be  issues  that 
affect  the  whole  organisation.  There  are  going  to  be 
times  when  Hangar  A  and  Hangar  B  have  the  same 
problems  and  it  would  be  a  waste  of  their  effort  if 
they  tried  to  resolve  these  individually.  Therefore,  we 
use  our  existing  “quality  forums”  to  monitor  MESH 
profiles  from  an  organisational  point  of  view.  In  the 
future  we  hope  to  combine  local  profiles  so  that  the 
highest  forum  in  our  organisation,  the  EQMRB 
(Engineering  Quality  Maintenance  Review  Board), 
can  actually  see  what  each  business  unit  (or 
department)  looks  like.  We  think  that  such 
information  will  be  very  useful  high  level  decisions. 

So,  what  do  we  have  planned  for  MESH  in  the 


Local  Process 

o  All  users  individually  trained 

o 

Monthly  MESH  Input  Roster 

© 

20%  sample  of  hangar  staff 

All  workshop  and  office  staff 

a  MESH 

Results 

Analysed  by  Champions  in  TQ  groups 

Issues  identified/prioritized  from  MESH  Profiles  and 

Comments 

© 

Actions  fed  back  to  users  -  newsletters,  noticeboards 

@ 

Past  resolutions  of  issues  monitored  for  effectiveness 
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future?  Well,  it’s  not  the  best  time  to  give  this 
presentation  at  this  moment  because  I’m  actually  in 
the  process  of  analysing  results  from  the  trial  over  the 
last  six  months,  and  will  be  presenting  my  findings  to 
our  Human  Factors  Steering  Group  next  month.  As  I 
stated  before  we  are  looking  at  a  number  of  software 
modifications  especially  to  the  Analysis  tool. 

We  are  also  looking  at  networking  the  system, 
because  at  the  moment  we  to  collect  MESH  data 
from  each  area  on  disk  and  combine  it  on  my  PC. 
Obviously,  that’s  not  an  efficient  way  of  doing  it. 
Also  once  MEDA  (Human  Factors  Investigation  tool) 
is  producing  results,  I  would  like  to  see  both  systems 


working  together  as  an  integrated  Human  Factors 
tool,  using  proactive  and  reactive  data. 

I  think  that  it’s  clear  that  we’ve  got  to  change  some 
of  the  traditional  engineering  attitudes  to  human 
error.  However,  I  think  there  is  one  engineering 
attitude  or  tradition  that  I  would  like  to  keep  and  that 
is  -  we  are  pretty  good  as  a  profession  at  turning 
theory  into  practice.  I  see  that  as  my  job,  turning 
Human  Factors  theory  into  practical  applications.  I 
believe  that  the  Human  Factors  Guide  that  you  have 
been  working  on  over  the  past  few  years  will  be 
useful  by  giving  practical  advice  on  a  misunderstood 
subject  and  I  believe  all  our  efforts  will  pay 
dividends  in  aircraft  safety,  quality  and  costs  in  the 
future.  Thank  you. 
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OVERVIEW  OF  FAA  NDI  RESEARCH 
AT  SANDIA  LABS 


Patrick  Walter,  PH.D, 
November  9, 1994 


Has  anyone  told  you  what  Sandia  means?  In 
Spanish,  it  translates  as  watermelon.  The 
mountain  range  to  the  north  is  the  Sandias.  The 
mountain  range  to  the  south  is  the  Monsontos,  which 
translates  as  apple.  When  the  sun’s  setting  at  the  right 
position,  the  mountains  turn  a  pinkish  hue  for  about 
thirty  seconds  every  night.  The  mountains  are  full  of 
deer,  mountain  lions,  coyote,  and  bear. 

Just  to  orient  you,  Sandia  National  Labs  is  a 
Department  of  Energy  Laboratory.  Sandia’ s  history 
has  been  one  of  designing  nuclear  weapons,  doing 
underground  nuclear  testing,  and  maintaining  the 
nuclear  weapons  stockpile.  Actually,  we’re  fairly 
busy  today.  One  of  the  things  we’re  doing  is  helping 
the  Russians  dismantle  their  stockpile  of  nuclear 
weapons.  There’s  a  segment  of  the  program  at  our 
laboratory  that  works  for  other  government  agencies. 
Sandia’ s  big  customer  is  the  Department  of  Defense. 
The  FAA  came  to  Sandia  after  Congress  passed  the 
1988  Aviation  Safety  Act  to  ask  if  we  would  get 
involved  with  the  inspection  portion  of  their  Aging 
Aircraft  Program.  We  did. 

Last  week,  we  hosted  a  meeting  both  of  the  ATA 
Inspection  Network  and  its  NDT  Forum  with  about 
four  hundred  people  attending,  including  fifty-one 
airlines  from  twenty-seven  countries.  What  I’m  going 
to  do  today  is  to  go  through  the  same  presentation  I 
gave  at  the  forum  a  week  ago  today.  There  are  a  lot  of 
human  factors  elements  that  tie  into  your  programs 
and  other  things  I  think  will  interest  you.  I  am  going 
to  deviate  from  my  previous  presentation  only  in  style 
to  point  out  things  that  I  believe  relate  to  your 
interests. 

Our  program’s  objective  is  to  provide  the  FAA  with  a 
tool  for  independent  and  quantitative  performance 
and  cost  assessments  of  aircraft  structural  inspection 
techniques.  If  you  want  to  introduce  new  inspection 
technologies  to  airlines  today,  economics  is  a  big  part 
of  the  process.  I  will  point  where  I  think  we  are 
saving  the  aviation  industry  some  dollars,  but  the 
program’s  overall  goal  is  to  encourage  NDI  technique 


development  and  validation.  We  have  test  beds  in  a 
hangar  that  you  are  going  to  visit  during  the  tour  this 
afternoon. 

I  am  not  going  to  give  you  a  lot  of  verbiage;  I  am 
going  to  show  you  some  pictures  of  our  work 
activities.  We  were  funded  in  September  1991,  and 
for  our  first  seventeen  months  we  were  essentially  a 
facility  organization.  We  acquired  a  hangar,  probably 
a  small  hangar  by  most  of  your  standards.  Our  hangar 
has  28,000  square  feet.  When  we  got  it,  the  roof  was 
caving  in  and  the  floor  would  not  hold  a  transport 
aircraft.  We  leased  the  hangar,  then  rehabbed  it  and 
acquired  test  beds.  We  installed  support  equipment, 
scaffolding,  and  a  catwalk  so  that  we  could  walk 
along  the  top  of  our  transport  airplane.  Since  the  FAA 
is  responsible  for  the  inspection  program  under  the 
National  Aging  Aircraft  Program,  we  worked  with 
them  to  define  some  program  infrastructure  so  that 
research  could  be  focused  towards  Albuquerque.  We 
wanted  essentially  to  function  as  the  hand-off  to  the 
aviation  industry,  so  we  defined  items  such  as  a 
validation  process  and  procedures.  In  February  1993, 
we  opened  our  facility  doors.  Soon  after,  researchers 
wanted  to  come  and  start  working  on  our  test  beds. 
People  came  from  industry  and  from  universities  with 
graduate  students,  so  we  had  to  next  implement  safety 
procedures. 

Researchers  were  making  a  lot  of  samples  of  flaws 
typically  occurring  in  aircraft  structure.  We  pooled  all 
the  information  together  and  archived  it.  If  FAA 
researchers  were  looking  for  specific  flaw  samples, 
we  could  tell  them  where  samples  were  available.  We 
formalized  an  FAA  Sample  Defect  Library  which  is 
published  and  routed  as  a  DOT  document.  We  also 
developed  a  database.  All  experimenters  acquire  data, 
and  the  common  result  of  most  advanced  inspection 
techniques  is  an  image.  Visual  inspection  is  not  about 
Just  a  flashlight  and  mirror;  in  its  broadest  sense,  it 
can  encompass  someone  looking  at  an  image  of  some 
NDT-determined  flaw  on  a  screen.  When 
experimenters  left  our  facility,  they  provided  floppy 
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disks  containing  inspection  results.  We  archived  these 
floppies  into  a  database. 

We  began  our  work  with  reliability  experiments.  I 
will  subsequently  tell  you  about  work  we  are  doing  in 
reliability  and  then  touch  on  human  factors.  We 
integrated  economics  into  our  program  early  on,  and 
an  economist  works  with  us.  We  are  initiating  a 
program  to  assess  results  of  experimentation  in  our 
facility  to  date.  Thus  far,  when  an  experimenter  has 
come  to  our  facility,  it  has  been  a  somewhat  informal 
process.  We  are  now  developing  rigorous  protocols 
and  acquiring  blind  samples  we  need  to  fill  our 
Sample  Defect  Library.  The  core  of  any  program  has 
to  be  blind  samples;  blind  implies  unknown  to  the 
experimenters.  We  also  work  for  the  FAA  in  areas 
such  as  information  systems  and  structural  activities. 
We  routinely  interact  with  Boeing  and  Douglas.  If 


you  are  here  from  an  airline,  chances  are,  I  have  been 
to  your  facility.  The  lab  is  going  to  get  formal 
industry  feedback  as  to  results  of  our  structured 
experiments. 

When  I  make  a  presentation  at  one  of  these  meetings, 
I  am  typically  asked:  “Where’s  the  beef?”  What  I 
want  to  do  is  to  show  you  some  things  we’ve 
completed  to  date.  In  November  1992,  we  got  our 
first  test  specimen  for  the  Sample  Defect  Library 
(Figure  A.l).  It  is  Sample  100  and  is  from  a  Boeing 
737.  It’s  from  the  same  series  as  the  Aloha,  which 
means  it  has  the  cold  bond  joints  and  that  terminating 
actions  have  not  been  performed  on  it.  It’s  relatively 
low  cycle,  46,000  cycles,  38,000  hours.  We  acquired 
it  from  Miranda,  Arizona,  and  it  was  flown  here  for 
its  last  flight. 
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Since  you’re  going  to  be  visiting  our  hangar  later  this 
afternoon,  I  can  share  a  little  humor  (Figure  A.2). 
When  we  acquired  the  Boeing  airplane,  there  was  still 
a  lot  of  emotion  from  the  Aloha  accident.  You  can 
imagine  how  Boeing  appreciated  that  there  was  a 
hangar  in  the  airport  for  Aging  Aircraft,  and  there  was 
a  Boeing  airplane.  Soon  after,  we  brought  in  some 
Douglas  DC-9  fuselage  sections  because  there  arc 
some  known  problems  around  the  bulkhead  and  with 
cracking  around  the  windows.  This  also  created  a 
balance  among  manufacturers. 

At  the  8th  FAA/AAM  Meeting  on  Human  Factors 
Issues  in  Aviation  Maintenance  last  year,  Floyd 
Spencer  presented  the  results  of  an  eddy  current 
reliability  experiment  to  you,  so  I  am  going  to 
mention  it  only  briefly  (Figure  A.3,  next  page).  When 
you  go  to  the  hangar,  you  are  going  to  see  the  panels. 
Floyd  is  going  to  be  standing  by  them  in  case  you 
have  any  questions.  One  question  that  came  up  after 
Aloha  was,  “How  well  is  inspection  being  done  in 
airline  maintenance  facilities?”  The  way  this  system 
works  is  that  fatigue  modeling  of  the  airframe  is 
performed.  Predictive  crack  row  models  are  used,  and 


assigned  inspection  intervals  must  be  frequent  enough 
for  one  to  find  a  crack  before  it  gets  to  a  critical 
length.  On  the  Aloha  plane’s  lap  splices,  cracks 
emanated  from  under  the  rivets,  and  so  the  question 
came  up,  “How  well  is  eddy  current  lap  splice 
inspection  being  done  in  aviation  maintenance 
facilities  today?”  No  one  had  ever  quantified  an 
answer  before.  The  U.  S.  Air  Force,  through  its  ‘have 
cracks,  will  travel’  program,  made  a  round  robin  in 
DOD  facilities  a  number  of  years  ago.  Typically, 
Boeing  and  Douglas  develops  a  predictive  probability 
of  detection  (POD)  model.  They  then  take  the  ninety 
percent  POD  and  shift  it  to  fifty  percent  to  include 
degradation  they  assume  will  incur  in  going  from 
their  facility  to  American  Airlines,  Tramco,  Dalfort, 
United  or  whomever.  They  degrade  their  curve 
conservatively  to  account  for  differences  between 
their  results  and  those  they  believe  will  occur  in 
maintenance  facilities. 

However,  the  question  is:  “How  good  is  industry 
doing?:  We  have  just  completed  a  $  1 .5  million 
experiment  for  the  FAA  during  which  we  visited  nine 
maintenance  facilities.  When  we  called  the  facilities, 
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they  really  welcomed  us.  The  concept  of  Sandia  Labs 
being  neutral  on  a  three-legged  stool-airline,  OEM, 
FAA-appears  to  have  been  accepted.  People  in  the 
field  clearly  distinguish  among  us,  the  FAA,  the 
OEMs,  and  the  airlines.  There  have  been  three  DOT 
volumes  published  supporting  this  experiment.  The 
last,  which  is  full  of  human  factors  information,  is  at 
the  FAA  Tech  Center  right  now  awaiting  publication. 


The  experimental  parameters  in  this  experiment 
include  a  Boeing  737  structure  with  enough 
inspection  sites  to  keep  an  inspector  working  six  to 
eight  hours,  day  shifts,  night  shifts,  crack  angles, 
painted  versus  unpainted  surfaces,  accessibility,  etc. 
We  went  into  a  facility,  set  up,  followed  developed 
protocols,  and  asked  the  inspector  to  pull  a  job  card 
and  go  to  work. 
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Within  the  program,  we  have  addressed  some 
structural  issues,  and  we  have  made  some  structural 
assessment  on  our  B-737  airframe.  I  can  tie  in  some 
human  factors  observations.  The  scanners  make  an  X- 
Y  scan;  depending  on  the  sensing  technique,  they 
paint  an  image  of  an  aircraft’s  structure  and  its 
condition  at  the  location  being  inspected  (Figure 
A.4).  Some  people  came  in  wearing  backpacks  and 
with  visual  displays.  Some  would  set  up  their 
equipment  and  after  ten  minutes,  it  would  be  on  the 
floor,  and  their  experiment  was  over.  Other  people 
were  successful  in  adapting  commercially  available 
equipment  to  airframe  structure.  Some  of  it  worked 
remarkably  well.  We  did  time  trials,  assessed  the 
efficacy  of  inspection  results,  etc.  Our  results  are 
available  as  a  DOT  report. 

As  a  byproduct  of  some  of  our  work,  the  Coast  Guard 
has  asked  to  join  our  program.  Their  HU25A  aircraft 
is  a  Falcon  20  derivative  (Figure  A.5,  next  page). 

This  particular  plane  did  a  lot  of  reconnaissance  over 
Haiti.  The  Coast  Guard,  like  everybody  else,  is 
running  short  of  money,  and  they  cannot  afford  to 
maintain  their  entire  fleet.  They  have  park  nine  of 


their  forty-one  planes,  so  they  donated  one  to  us.  The 
plane  is  made  by  Dessault.  The  American 
representative  is  Falcon  Jet.  There’s  an  SSIP 
(Supplemental  Structural  Inspection  Program) 
coming  on  this  aircraft,  and  SSIP’s  draft  indicates  that 
it  will  cost  the  Coast  Guard  a  large  sum  of  money. 
Dessault  has  met  with  us,  and  we  think  there  is  real 
potential  for  knocking  the  cost  down  by  integrating 
some  advanced  inspection  techniques. 

As  I  mentioned,  we  defined  the  deliverable  from  our 
validation  process  to  be  an  inspection  technique’s 
reliability  and  cost-effectiveness.  When  we  went 
around  the  country  to  various  maintenance  facilities 
with  our  eddy  current  inspection  experiment,  people 
used  their  own  inspection  equipment  in-house.  For 
example,  when  we  went  to  United,  we  knocked  on  the 
door  and  asked  if  we  could  come  in  and  take  hangar 
space  and  use  your  inspectors  on  day  shifty,  night 
shifts,  etc.  We  said  the  we  would  need  forklifts  and 
that  it  really  would  be  an  inconvenience.  To  their 
credit,  they  said  that  it  was  okay,  that  they  wanted  us 
there,  and  that  they  would  give  us  a  week’s  worth  of 
time,  as  we  needed  it.  And  then  we  said,  “Well,  listen, 
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let  us  ask  you  something  else.  Here’s  a  new  piece  of 
equipment,  the  MOL  It’s  designed  to  do  the  same 
type  of  inspection  work  as  the  eddy  current 
equipment,  but  it  works  a  little  differently.  Instead  of 
looking  at  a  meter  on  impedance  plane,  it  gives  a 
visual  image  of  the  magnetic  field.  It  might  have 
some  advantages  in  terms  of  reliability  or  cost- 
effectiveness.  If  you  want,  we  will  provide  a  one-day 
training  program.  Then  you  can  try  it  for  another 
week,  but  then  you  will  have  to  repeat  the  whole 
experiment,  go  through  this  whole  process  again,  so 
now  it  is  going  to  take  two  weeks  of  your  time.” 

Without  exception  United  and  everyone  else  said, 
“We  want  to  give  you  another  week’s  worth  of  our 
time.”  We  put  together  a  training  program.  They 
repeated  the  experiments,  and  we  acquired  field 
reliability  for  this  instrument  to  compare  against  the 
existing  eddy  current  technique.  We  then  worked  with 
the  Transportation  Center  at  Northwestern  University 
to  perform  a  cost-benefit  analysis.  We  presented  it  at 
the  ATA  meeting,  and  a  report  is  coming  out  on  this 
now.  The  report  shows  that  if  you  can  defer  paint 
stripping,  you  can  come  up  with  some  non-trivial 
economic  advantages. 


Another  question  that  came  up  in  using  the  B-737  as 
a  test  bed  was,  “What  was  its  condition?”  We 
transformed  our  facility  into  a  maintenance  facility 
and  performed  a  heavy  check  on  that  airplane.  Our 
procedure  was  to  pick  out  structural  elements  of  the 
heavy  check  that  we  thought  we  needed  to 
incorporate  into  our  assessment,  to  initiate  a  cost 
estimate,  and  to  identify  the  SIDS  and 
supplementation  inspections  we  wanted  to 
incorporate.  Then,  we  contacted  some  ex-airline 
employees  who  comprised  a  consulting  company-ex- 
United,  FedEx,  and  Continental  folks-and  brought 
them  in  The  group  included  a  retired  foreman  who 
had  thirty  years’  experience.  This  company  was  Tech 
Ops.  We  acquired  visual  inspectors  from 
Albuquerque  Airport;  they  had  twenty  to  twenty-five 
years’  experience  and  had  bounced  around  in  the 
industry,  spending  their  career  working  on  transport 
aircraft.  Then,  we  went  down  to  Miami  and  hired  a 
commercial  NDT  firm,  Miami  NDT,  who  contracts  to 
third-party  maintenance  facilities  and  has  a  large 
presence  in  NDT  inspections  in  the  Miami  area.  We 
brought  all  these  people  together,  transformed  one  of 
our  rooms  into  a  maintenance  planning  room,  put 
together  a  hundred  and  thirty-five  job  cards,  and 
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performed  what  we  call  a  baseline  of  that  airplane. 
This  means  that  we  inspected  as  well  as  can  be  done 
with  the  tools  present  in  civil  aviation  today.  This 
means  a  lot  of  visual  inspection  and  a  little  bit  of 
fairly  simple  NDT.  Tech  Ops  held  the  “baton”  during 
the  entire  process. 

Over  the  past  year,  fifty  groups  have  worked  in  the 
facility.  When  you  go  over  today,  there  will  be  two 
groups  working,  and  those  of  you  that  have  been  in 
the  inspection  industry  will  be  kind  of  interested. 
NASA  Langley  has  four  people  here  doing  thermal 
wave  imaging  by  putting  heat  pulses  into  airplanes 
and  looking  at  temperature  differences  on  the  surface 
to  detect  corrosion  and  disbonds.  They’re  also  doing 
some  ultrasonic  work.  There  is  also  a  commercial 
firm.  Holographies,  Inc.,  who  are  putting  some 
vibratory  inputs  into  our  aircraft.  They  are  able  to  use 
laser  detection  for  things  like  subsurface  structural 
damage,  cracked  frames,  stringers,  clips  missing,  etc. 
They  want  to  rehearse  this  technique  and  ultimately 
give  the  FAA  and  industry  the  opportunity  to  assess 
it. 

The  FAA,  in  setting  up  our  center,  also  funded  a 
number  of  other  locations  across  the  country.  They 
put  up  two  principal  centers,  the  one  here  in 
Albuquerque  and  a  university  consortium  at  Iowa 
State  University.  The  other  researchers  undertake 
basic  and  applied  research,  and  we  have  the  role  of 
technology  transfer  to  industry,  i.e.,  the  industry 
hand-off.  Based  on  industry  feedback  and  FAA 
review,  nine  tasks  at  Iowa  State’s  consortium  look 
like  they  have  the  potential  to  contribute  to  the 
aviation  industry.  We’re  working  with  them  to  make 
working  prototypes  that  we  can  demonstrate  to 
industry  in  our  center.  Industry  then  has  the 
opportunity  to  express  its  interest  and  state  whether  it 
wants  this  research  to  proceed. 

I  don’t  know  how  many  of  you  have  heaid  about 
widespread  fatigue  damage,  but  it  is  a  topic  a  number 
of  folks  are  actively  discussing  right  now.  It  involves 
small  generalized  cracking  in  airframes  and  has  the 
potential  to  lower  an  airframe’s  capability  to  be 
damage-tolerant.  I’m  not  here  to  debate  that  issue. 
TOGO  (Technical  Oversight  Group  On  Aging 
Aircraft)  an  FAA/industry  review  group,  has  this 
issue  as  one  of  their  concerns.  We  have  been  asked  to 
be  able  better  to  assess  technologies  for  detecting 
small  cracks  in  airframes.  The  typical  crack,  at  least 
around  a  rivet,  an  inspector  looks  for  now  is 
nominally  a  tenth  of  an  inch,  but  interest  has  been 
expressed  in  looking  at  cracks  in  the  forty  and  fifty 


mil  range.  To  reach  that  goal,  we  need  both  blind 
specimens  and  industry  and  FAA  researchers  coming 
in  to  have  their  technologies  assessed.  Every  task  we 
are  working  on  has  an  industry  linkage.  Concerning 
subsurface  flaws,  i.e.,  cracking  in  second  and  third 
layer  structures,  a  requirement  again  exists  for  blind 
samples.  We  are  working  cooperatively  with  Boeing 
and  Douglas  to  design  those  samples,  and  Northrop  is 
also  a  player.  Northrop  represents  the  military  side, 
but  they  also  do  R&D  on  improved  inspection 
techniques.  It  looks  like  their  technology  will  well 
lend  itself  to  the  civilian  side. 

As  you  are  aware,  to  inspect  commercial  transports, 
you  open  them  up  and  go  in  with  a  flashlight  and  a 
mirror  to  visually  inspect  for  corrosion.  The  Air 
Force  is  actively  pursuing  wide  area  and  local 
nondestructive  inspection  techniques  to  detect 
corrosion.  It  is  tremendous  what  service  lives  are 
predicted  for  Air  Force  planes.  For  example,  the  KC 
135  has  to  fly  until  the  year  2030.  Their  cycle  time  is 
low,  and  the  fatigue  problem  is  not  accentuated  if  you 
can  prevent  corrosion.  The  Air  Force  places  a  lot  of 
emphasis  on  corrosion  detection.  We  are  leveraged 
with  them  for  cooperation  between  our  FAA  program 
and  their  Air  Force  program. 

Visual  experimentation  is  another  activity  at  our 
facility.  We  have  continuous  requests  to  assess  how 
well  visual  experimentation  occurs  in  industry  today. 
Colin  Drury,  who  spoke  earlier,  is  working  on  this 
task  with  us.  On  the  earlier  eddy  current  reliability 
experiment  that  I  talked  about,  we  were  linked  to  the 
0AM  through  Bob  Blanchard  over  at  CAMI.  We 
have  tried  to  keep  an  involvement  with  this  audience. 
Concerning  the  visual  experimentation,  the  ATA  (Air 
Transport  Association)  has  appointed  an  advisory 
group,  including  five  airline  people.  We’re  planning  a 
program  over  the  next  four  to  six  months  so  that  we 
should  have  between  twenty-four  and  thirty-six  airline 
inspectors  through  our  facility.  We  are  putting 
together  a  standard  visual  inspector’s  tool  box  to 
normalize  the  inspectors’  tools,  and  we  will  complete 
experimentation  on  the  transport  aircraft  before 
moving  into  the  commuter  area  over  the  next  year. 

We  appreciate  the  ATA’s  support. 

Halon  bottles  are  another  interesting  topic.  The  ATA 
has  asked  us  to  work  in  this  area.  American  recently 
estimated  a  $10  million  dollar  savings  for  the  industry 
in  the  next  year  if  we  are  successful.  Halon  bottles  in 
aircraft  must  be  hydro-tested:  there  is  a  DOT 
regulation  about  it.  The  bottles  are  collected  and  sent 
to  one  of  three  centralized  locations.  The  halon  is 
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captured,  and  the  bottles  are  hydro-tested,  reloaded, 
shipped  back  to  the  airline,  and  reinstalled.  We  think 
it’s  going  to  be-I  don’t  want  to  say  easy-certainly  not 
difficult  to  heat  the  bottle  by  about  forty  degrees  and 
to  avoid  disassembly  and  shipping  by  looking  at 
acoustic  emissions. 

A  DC-9  wingbox  inspection  is  another  work  activity. 
When  I  first  became  involved  with  this  program,  I 
went  to  an  inspection  facility  and  worked  with 
inspectors  for  several  days,  crawling  through  the 
airplanes.  The  head  of  maintenance  told  me  that  if  I 
wanted  to  serve  the  industry,  I  needed  to  assume  that 
the  average  inspector  has  a  minimum  of  a  ninth-grade 
education  and  wants  to  do  a  good  job.  We  have  to 
design  tools  for  the  inspector.  I  believe  another 
variable  is  if  one  can  save  someone  enough  money, 
some  constraints  can  lessen.  In  the  case  of  the  DC-9, 
Northwest  Airlines  expressed  interest.  In  the  DC-9’s 
wingbox,  there  is  a  potential  problem  due  to  stress, 
corrosion,  and  cracking.  The  inspection  procedure 
involves  an  extensive  teardown.  If  the  wingbox  T-cap 
is  in  good  shape,  the  unit  can  be  reassembled.  By 
working  cooperatively  with  the  technique’s  inventor, 
Northwestern  University,  we  think  we  have  an 
ultrasonic  technique  that  can  potentially  replace  a 
teardown  with  a  forty-hour  inspection,  a  savings  of 
hundreds  of  hours  per  airplane.  Since  Northwest  has  a 
fleet  of  something  like  a  109  DC-9s,  potential  savings 
are  in  the  millions  of  dollars.  We  hope  soon  to  have  a 
service  bulletin  revision  from  Douglas,  who  is  fully 
cooperating  in  this  project. 

The  Air  Force  uses  boron  epoxy  bonded  repairs  on  a 
lot  of  their  planes.  Boron  epoxy  has  some  unique 
properties.  It  is  directional  in  nature  and  has  very  high 
stiffness.  We  were  interested  in  supporting  the  FAA 
by  doing  some  work  with  boron  epoxy,  and  the  FAA 
supported  a  workshop  at  our  facility.  There  were 
concerns  about  who  would  be  the  initial  customer. 
Delta  volunteered,  and  Lockheed  expressed  interest 


in  issuing  a  service  bulletin  concerning  the  use  of 
boron  epoxy  as  a  structural  reinforcement.  We  now 
have  a  joint  program  with  Delta  and  Lockheed  on  the 
L-101 1.  Our  parking  lot  at  the  hangar  has  some  big 
slabs  of  L- 101 1  we  have  cut  out  and  hauled  in.  We 
are  also  working  cooperatively  with  the  Atlanta  AGO. 
Textron  is  the  boron  epoxy  supplier,  and  Warner 
Robbins  AFB  is  also  pai'ticipating. 

Most  of  the  program’s  focus  to  date  has  been  on 
transports.  Commuters  can  certainly  benefit  from  this 
program,  especially  since  some  commuters  contain 
inaccessible  structure.  We  are  going  to  bring  a 
commuter  airplane  into  our  hangar  in  the  next  six 
months.  By  the  time  we  finish  the  visual  inspection  on 
the  transport,  we  will  be  ready  to  move  to  the 
commuter  and  use  it  as  a  test  bed  for  other  activities. 

Candidly,  when  we  started  this  program  a  few  years 
ago,  we  thought  we  had  some  tools  to  contribute,  but 
the  nature  of  that  contribution  was  not  totally  clear. 
Now,  we  feel  like  we  are  delivering  products  to  the 
aviation  industry.  I  do  not  say  that  out  of  arrogance, 
but  because  the  industry  is  indicating  that  we  are 
contributing.  The  FAA  is  negotiating  with  us  to 
expand  our  program  and  to  acquire  more  access  to 
other  areas  of  Sandia. 

We  feel  we  are  coordinating  well  with  Douglas  and 
Boeing.  As  time  goes  by,  we  hope  to  work  more  with 
GE  and  Pratt  to  place  more  emphasis  on  engines  in 
our  program.  We  feel  that  we  work  well  with 
individual  airlines.  We  routinely  have  airline 
personnel  in  our  facility,  and  we  encourage  and 
appreciate  that.  What  we  would  like  to  do  is  to  set  up 
a  more  structured  interface  with  the  ATA  and  are 
talking  with  them  about  how  we  might  develop  an 
optimal  interface.  We  need  to  get  more  involved  in 
commuter  issues;  our  commuter  aircraft  is  coming  in. 
As  I  told  participants  at  the  ATA  Forum,  we’re  trying 
to  listen  to  everybody,  and  it  is  a  big  industry. 
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